Systems and methods for multi-resolution image processing

ABSTRACT

Systems and methods for the processing of images over bandwidth-limited transmission interfaces, such as processing of high resolution video images over standard analog video interfaces, using multi-stream and/or multi-resolution analog methodology. The disclosed systems and methods may also be implemented to provide video resolutions across standard analog video interfaces in a manner that supports digital capture and imaging techniques to obtain useful information from digitally zoomed and enhanced video.

[0001] This patent application claims priority to copending U.S.Provisional patent application Ser. No. 60/456,294, filed Mar. 20, 2003,and entitled “Systems And Methods For Creation, Transmission, AndViewing Of Multi-Resolution Video” by Washington, the entire disclosureof which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

[0002] The present invention relates generally to video systems, and inone exemplary application to multi-image video systems.

[0003] Existing closed circuit television (“CCTV”) analog videotransmission-based surveillance installations typically employ ITU/ISONTSC (720×480) or PAL (720×576) standard video resolutions that arelimited by realizable system bandwidth and cost constraints associatedwith such installations. FIG. 1 depicts an example of such a prior artCCTV installation 100 including a standard resolution (i.e., NTSC orPAL) digital camera 102 that is connected to a multi-port personalcomputer-based digital video recorder (“DVR”) 104 through abandwidth-limited analog transmission interface 106 to provide astandard resolution displayed image 108.

[0004] Factors that limit the maximum bandwidth of such prior artinstallations include signal attenuation associated with requiredmaximum installation cabling length (typically up to 1000 feet), thetype of cabling used (typically standard RG59 coax) and standardinstallation techniques that can result in non-optimal terminations.Increased costs associated with use of improved cabling, improvedinstallation techniques, and/or custom interface circuitry precludetheir use to achieve increased bandwidth capability in many situations.Transmission of higher resolution video imagery may be accomplishedusing digital transmission technology, but requires additionalinvestments for new higher cost equipment, including compression andtransport circuitry.

SUMMARY OF THE INVENTION

[0005] Disclosed herein are systems and methods that may be employed forthe processing (e.g., creation, transmission and/or reconstruction) of adesired video scene (e.g., CCTV scene). In one embodiment, the disclosedsystems and methods may be employed for the analog transmission of avideo scene, including transmission of the desired scene over standardanalog interfaces (e.g., standard CCTV interfaces), and for the displayof the desired scene simultaneously in multiple resolutions (e.g., highand standard resolutions simultaneously), and/or for the display ofmultiple images of the desired scene on an analog display (e.g.conventional analog composite video monitor), using multi-stream (e.g.,dual-stream analog) methodology.

[0006] Advantageously, the disclosed systems and methods may beimplemented in one embodiment to fulfill the need forhigher-than-standard video resolution imaging for video surveillanceinstallations using existing digital video equipment (e.g., usingexisting standard digital to analog encoders, existing coax connections,existing chipsets, existing standard analog to digital de-encoders,etc.) with modified or new software capable of implementing one or morefeatures of the disclosed systems and methods (e.g., in one exemplaryembodiment by modifying Windows-based DVR software on PC or Intel-basedDVR systems). Using the disclosed systems and methods CCTV videoresolutions may be provided that support and take advantage of digitalcapture and imaging techniques to obtain useful information fromdigitally zoomed and enhanced video.

[0007] In another embodiment, the disclosed systems and methods may beimplemented to generate multiple camera views from a relatively higherresolution image source (e.g., high resolution image sensor, imagestorage device containing high resolution image data, etc.) fortransmission and display over relatively lower resolution video mediacomponents (e.g., standard video media such as across a standard analogcomposite video interface for display on a standard analog compositevideo display monitor). In such an embodiment, multiple imageinformation streams may be integrated into a standard definition videostream as embedded camera views and/or may be sent as individual videostreams, simultaneously over additional video media. In one exemplaryembodiment, a single camera or video image storage device may beemployed to virtualize and emulate multiple image sources (e.g.,multiple cameras) with individual viewing and control capabilities.

[0008] As used herein, “high resolution” may be characterized as a videoresolution that is greater than standard NTSC or PAL resolutions.Therefore, in one embodiment the disclosed systems and methods may beimplemented to provide a resolution greater than standard NTSC andstandard PAL resolutions, or greater than 720×576 pixels (414,720pixels, or greater), across a standard composite video analog interfacesuch as standard coaxial cable. Examples of some common high resolutiondimensions include, but are not limited to: 800×600, 852×640, 1024×768,1280×720, 1280×960, 1280×1024, 1440×1050, 1440×1080, 1600×1200,1920×1080, and 2048×2048. In another embodiment, the disclosed systemsand methods may be implemented to provide a resolution greater thanabout 800×600 pixels (i.e., 480,000 pixels), alternatively to provide aresolution greater than about 1024×768 pixels, and further alternativelyto provide HDTV resolutions of 1280×720 or 1920×1080 across a standardcomposite video analog interface such as standard coaxial cable.Examples of high definition standards of 800×600 or greater that may beso implemented in certain embodiments of the disclosed systems andmethods include, but are not limited to, consumer and PC-based digitalimaging standards such as SVGA, XGA, SXGA, etc. It will be understoodthat the forgoing examples are representative of exemplary embodimentsonly and that the disclosed systems and methods may be implemented toprovide enhanced resolution that is greater than the native or standardresolution capability of a given video system, regardless of theparticular combination of image source resolution and type of interface.

[0009] In another embodiment, high resolution image source/s (e.g.,1024×768 pixel range or greater) may be processed in a fashion thatgenerates multiple output video streams, that may be used to provideseparate viewing capabilities. This may be accomplished, for example, bytaking the video source output (e.g., image sensor based video output)which may be a stream of video ‘frames’, and generating (eitherphysically or logically) multiple video output streams to individualprocessing units. These video output streams may all be identical, oralternatively may be samplings of different visual regions of the outputvideo frames of the image source. It will be understood that the actualvisual content of each video stream may vary according to the needs ordesired characteristics for a given application, and is independent ofthe implementation of one or more features of the disclosed systems andmethods.

[0010] In the practice of the disclosed systems and methods, examples ofadvantages that may be realized (e.g., alone or in any combination)using the disclosed systems and methods to provide higher resolutionvideo and/or multiple-resolutions for video surveillance or other videosignal processing applications include, but are not limited to, thefollowing. Higher resolution and multiple resolution video of thedisclosed systems and methods may be implemented to allow for areduction in the number of cameras required to cover an area ofinterest. In one example, use of HDTV resolutions may allow for areduction in the number of cameras (e.g., of a video surveillancesystem) by a ratio approaching or equal to 4:1 since the HDTV horizontalresolution is roughly 4 times that of standard definition cameras.Stored content associated with higher resolution video of the disclosedsystems and methods may also be taken advantage of to provide moreuseful information. For example, in most cases involving the occurrenceof an event of interest (e.g., occurrence of a security event such astheft, break-in, assault, etc.), cameras are often not zoomed into thecorrect space or given spatial portion of the video image to capture themost relevant or detailed data associated with the event. However,higher resolution information captured using the disclosed systems andmethods may be used to provide substantially increased detail of a givenportion of a video image and to provide greater information as to whatwas occurring at the time around an event of interest, even when thevideo camera is not zoomed into that give portion of the video image.Multiple resolution images of the disclosed systems and methods may alsobe used to provide the capability to maintain an overall awareness of ageneral situation while allowing simultaneous focus/zoom on a specificarea/s of activity or interest.

[0011] In one embodiment, a multi-resolution video stream may begenerated over a standard closed circuit television interface utilizinga high resolution sensor. The high resolution images from this sensor,may be scaled down to standard NTSC or PAL resolution and transmitted ata frame rate slower than that normally associated with NTSC or PAL framerates (e.g., 25 to 30 frames per second). Frame periods not occupied bythe scaled video images may then be used to transmit a tiled version ofthe full resolution image along with embedded information that allowsthe full high resolution image to be reconstructed without visibleartifacts. Image reconstruction may be performed by a personal computer,processor-based DVR, or other suitable processor or dedicated hardware,and the resultant high resolution image made available for display alongwith the lower resolution image on a high resolution monitor and/or madeavailable for digital storage. Using the embedded information alsoallows transmission of non-tiled versions of the original highresolution image intermixed with the tiled information. Examples wouldbe specific areas of the original high resolution image that have beenscaled, the entire original image scaled to a standard definition size,etc. In this way both a blend of lower frame rate high resolution imagesand higher frame rate standard resolution images may be sent across asingle interface

[0012] In such an embodiment, the disclosed systems and methods may beimplemented in a manner compatible with reduced frame ratecharacteristics of existing CCTV video surveillance industry technology.In this regard, typical digital video recorders provide a capability ofless than 30 frames/sec for video storage or display, with the averageframe rates utilized by the industry being in the range of from about 3to about 8 frames/sec. In the video surveillance industry, this framerate has been considered to provide an acceptable tradeoff betweensituational awareness, video storage size, cost, and processing powerrequirements. Therefore, high resolution images at these lower framerates may be acceptable, and/or high resolution images at even lowerframe rates may suffice for some situations where lower resolution(e.g., standard resolution images) are simultaneously available athigher frame rates. Although higher frame rates may be desirable forsome applications (e.g., such as trying to capture illegalsleight-of-hand techniques in gambling or theft situations), it has beenestimated that even these situations require only from about 15 to about18 frames/sec. In this regard, the disclosed systems and methods may beimplemented in one exemplary embodiment to provide standard resolutionimages within the range of from about 15 to about 30 frames/sec whilesimultaneously sending tiled images for subsequent reconstruction.

[0013] In another embodiment, the disclosed systems and methods may beimplemented to take advantage of the unused bandwidth associated withdiscarded or “dropped” frames of video that occur, for example, when agiven frame rate (e.g., 30 frames/sec) is available for transmission,but only a portion (e.g., about 2 to 7.5 frames/sec) of the given framerate is stored and/or displayed. This unused bandwidth may be utilizedto transmit higher resolution images, multiple resolution images,combinations thereof, etc.

[0014] In various embodiments of the disclosed systems and methods, anumber of exemplary features may be advantageously implemented, alone orin combination, to create, transmit, and/or view video scenes. Examplesof such exemplary features include, but are not limited to, creation ofa multiple resolution video stream compatible with standard CCTVinterfaces; transmission of a multiple resolution video streamcompatible with standard CCTV interfaces; utilization of patterns withinunused portions of a video frame for proper reconstruction of higherresolution images to reduce or eliminate visual artifacts; andutilization of patterns within unused portions of a video frame formarking of image segments as to their original spatial relationship andsubsequent detection and reaction to these markings.

[0015] In the practice of the disclosed systems and methods, one or moreportions of an image produced by an image source (e.g., image sensor,image storage device, etc.) may be zoomed before or after transmissionacross an analog interface. For example, in one embodiment one or morezoomed portions of an image received from an image source may begenerated prior to transmission in real time across an analog interfacefor analog display with no further processing, i.e., as zoomed image/sproduced and then transmitted as part of analog video signals across ananalog interface by multi-stream image processing circuitry of a camera.In another embodiment, one or more portions of an image may be digitallyprocessed and zoomed in real time after receipt of the image from acrossan analog interface (e.g., after transmission of tiled images across ananalog interface and reconstruction of a high resolution image therefromin a PC-based DVR). In yet another embodiment, image information may befirst stored after receipt across an analog interface (e.g., with orwithout further digital processing to reconstruct a high resolution orother type image from multiple tiles), and then retrieved and furtherdigitally processed as appropriate to produce one or more desired zoomedimages (e.g., in a PC-based DVR or using any other suitable type of DVRor circuitry suitable for this purpose). The term “zoomed” herein refersto the spatial scaling of all or part of an original image. In thisregard, the scaling factor may range in value from less than 1(reduction) and into positive factors greater than 1 (enlargement).

[0016] In one respect, disclosed herein is a method of providingmultiple images for transmission across an analog interface, including:receiving at least one digital image data input stream, the digitalimage data input stream containing digital image information; creatingat least two digital image data streams from the at least one digitaldata input stream, each of the at least two digital image data streamsincluding at least a portion of the digital image information; mergingthe at least two digital image data streams into a common digital imagedata output stream; converting the common digital image data outputstream into an analog image output stream; and providing the analogoutput image stream for transmission across the analog interface.

[0017] In another respect, disclosed herein is a method of providingmultiple images for transmission across an analog interface, including:receiving at least one digital image data input stream, the digitalimage data input stream containing digital image information; creatingat least two digital image data streams from the at least one digitaldata input stream, each of the at least two digital image data streamsincluding at least a portion of the digital image information, andwherein a first one of the at least two digital image data streamsincludes a first image may have a first resolution, and wherein a secondone of the at least two digital image data streams includes a secondimage may have a second resolution; converting a first one of the atleast two digital image data streams into a first analog image outputstream, and converting a second one of the at least two digital imagedata streams into a second analog image output stream; communicating thefirst analog output image stream across a first analog interface, andcommunicating the second analog output image stream across a secondanalog interface; receiving the first analog output image stream fromacross the first analog interface, the first analog output image streamincluding the first image; receiving the second analog output imagestream from across the second analog interface, the second analog outputimage stream including the second image; displaying the first image ofthe first analog output image stream on a first analog display device;and displaying the second image of the second analog output image streamon a second analog display device.

[0018] In another respect, disclosed herein is a method of processingdigital image data, including: providing the digital image data, thedigital image data including a full image; processing the digital imagedata in a first processing operation to create first processed imagedata including a first image; processing the digital image data in asecond processing operation to create second processed image dataincluding a second image; merging the first and second processed imagedata into a common merged data stream; converting the merged data streaminto a composite analog video stream containing the first and secondimages; and providing the composite analog video stream forcommunication across an analog interface. The first processed image datamay have an image resolution that is different from an image resolutionof the second processed image data, or the first processed image datamay include a different portion of the digital image data than thesecond processed image data, or a combination thereof.

[0019] In another respect, disclosed herein is multiple resolution imagecreation circuitry configured to receive at least one digital image datainput stream containing digital information, the multiple resolutionimage creation circuitry including: multi-resolution image processingcircuitry configured to: create at least two digital image data streamsfrom the at least one digital data input stream, each of the at leasttwo digital image data streams including at least a portion of thedigital image information, merge the at least two digital image datastreams into a common digital image data output stream; and conversioncircuitry coupled to the multi-resolution image processing circuitry andconfigured to: receive the common digital image data output stream fromthe multi-resolution image processing circuitry, convert the commondigital image data output stream into an analog image output stream, andprovide the analog output image stream for transmission across an analoginterface. A multiple resolution image creation and display system mayinclude the multiple resolution image creation circuitry, and mayfurther include an analog display device coupled to the analog interfacethat is configured to receive the analog video signal from across theanalog interface and to simultaneously display the first and secondimages of the analog video signal.

[0020] In another respect, disclosed herein is multiple resolution imagecreation and display system, including multi-resolution image processingcircuitry, conversion circuitry, and first and second display devices.The multi-resolution image processing circuitry may be configured to:receive at least one digital image data input stream, the digital imagedata input stream containing digital image information, create at leasttwo digital image data streams from the at least one digital data inputstream, each of the at least two digital image data streams including atleast a portion of the digital image information, and wherein a firstone of the at least two digital image data streams includes a firstimage may have a first resolution, and wherein a second one of the atleast two digital image data streams includes a second image may have asecond resolution. The conversion circuitry may be coupled to themulti-resolution image processing circuitry and be configured to:convert a first one of the at least two digital image data streams intoa first analog image output stream, and convert a second one of the atleast two digital image data streams into a second analog image outputstream, communicate the first analog output image stream across a firstanalog interface, and communicate the second analog output image streamacross a second analog interface. The first display device may becoupled to the first analog interface and may be configured to receivethe first analog output image stream including the first image fromacross the first analog interface, and to display the first image. Thesecond display device may be coupled to the second analog interface andmay be configured to receive the second analog output image streamincluding the second image from across the second analog interface, andto display the second image.

[0021] In another respect, disclosed herein is multiple resolution imagecreation circuitry for processing digital image data including a fullimage, the multiple resolution image creation circuitry includingmulti-resolution image processing circuitry, and conversion circuitry.The multi-resolution image processing circuitry may be configured to:process the digital image data in a first processing operation to createfirst processed image data including a first image, process the digitalimage data in a second processing operation to create second processedimage data including a second image, and merge the first and secondprocessed image data into a common merged data stream. The conversioncircuitry may be configured to: convert the merged data stream into acomposite analog video stream containing the first and second images,and provide the composite analog video stream for communication acrossan analog interface. The first processed image data may have an imageresolution that is different from an image resolution of the secondprocessed image data, or the first processed image data may include adifferent portion of the digital image data than the second processedimage data, or a combination thereof. A multiple resolution imagecreation and display system may include the multiple resolution imagecreation circuitry, and may further include an analog display devicecoupled to the analog interface and configured to: receive the compositeanalog video stream containing the first and second images from acrossthe interface; and simultaneously display the first and second imagescontained in the composite analog video stream.

[0022] In another respect, disclosed herein is an image processing anddisplay system including multiple resolution image creation circuitrycoupled to at least one analog display device by at least one analoginterface.

[0023] In another respect, disclosed herein is a method of providingmultiple images for transmission across a digital interface, including:receiving at least one digital image data input stream, the digitalimage data input stream containing digital image information; creatingat least two digital image data streams from the at least one digitaldata input stream, each of the at least two digital image data streamsincluding at least a portion of the digital image information; mergingthe at least two digital image data streams into a common digital imagedata output stream; providing the common digital image data outputstream for transmission across the digital interface.

[0024] In another respect, disclosed herein is multiple resolution imagecreation circuitry configured to receive at least one digital image datainput stream containing digital information, the multiple resolutionimage creation circuitry including multi-resolution image processingcircuitry configured to: create at least two digital image data streamsfrom the at least one digital data input stream, each of the at leasttwo digital image data streams including at least a portion of thedigital image information, merge the at least two digital image datastreams into a common digital image data output stream; and provide thecommon digital image data output stream for transmission across adigital interface. A multiple resolution image creation and displaysystem may include the multiple resolution image creation circuitry, andmay further include a digital processing device coupled to the digitalinterface and configured to receive the common digital image data outputstream from across the digital interface and to store the common digitalimage data output stream, to display one or more images contained in thecommon digital image data output stream, or a combination thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

[0025]FIG. 1 is a simplified schematic of a prior art standardresolution CCTV video surveillance system.

[0026]FIG. 2 is a simplified schematic of a multiple resolution videosystem according to one embodiment of the disclosed systems and methods.

[0027]FIG. 3A is a simplified schematic of a video system according toone embodiment of the disclosed systems and methods.

[0028]FIG. 3B is a simplified schematic of a video system according toone embodiment of the disclosed systems and methods.

[0029]FIG. 4A is a representation of four individual image tilesembedded in video frames as they may be oriented for reassembly into areconstructed image according to one embodiment of the disclosed systemsand methods.

[0030]FIG. 4B illustrates correlation of an alignment pattern accordingto one embodiment of the disclosed systems and methods.

[0031]FIG. 4C illustrates blending of tile overlap areas according toone embodiment of the disclosed systems and methods.

[0032]FIG. 4D represents a reconstructed image according to oneembodiment of the disclosed systems and methods.

[0033]FIG. 5A-5B represents selectable extraction of higher rate highresolution image area out of a lower rate high resolution imageaccording to one embodiment of the disclosed systems and methods.

[0034]FIG. 5C-5D represents selectable extraction of high resolutionimage area out of a larger standard resolution image according to oneembodiment of the disclosed systems and methods.

[0035]FIG. 6A represents an non-zoomed stored image taken by a videosurveillance camera.

[0036]FIG. 6B represents a digitally zoomed image of the non-zoomedstored image of FIG. 6A obtained when starting with a standardresolution image.

[0037]FIG. 6C represents a digitally zoomed image of the non-zoomedstored image of FIG. 6A obtained when starting with a high resolutionimage according to one embodiment of the disclosed systems and methods.

[0038]FIG. 7 represents selectable extraction, scaling, and display ofimage areas out of a higher resolution image according to one embodimentof the disclosed systems and methods.

[0039]FIG. 8A is a diagram showing image data logic flow according toone embodiment of the disclosed systems and methods.

[0040]FIG. 8B is a diagram showing image data logic flow according toone embodiment of the disclosed systems and methods.

[0041]FIG. 9 illustrates a de-interlaced active frame according to oneembodiment of the disclosed systems and methods.

[0042]FIG. 10 illustrates placement of a tile identifier pattern in avertical frame according to one embodiment of the disclosed systems andmethods.

[0043]FIG. 11A-11D illustrates tile identifier patterns according to oneembodiment of the disclosed systems and methods.

[0044]FIG. 12 illustrates display of multiple images according to oneembodiment of the disclosed systems and methods.

[0045]FIG. 13 illustrates image reconstruction methodology according toone embodiment of the disclosed systems and methods.

[0046]FIG. 14 illustrated selection of a variable sized subset of alarger resolution image and positioning of the selection area within thelarger image in relationship to external pan and tilt commands.

DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

[0047] The disclosed systems and methods may be implemented to allowimage information that originates from one or more image sources to beprocessed and provided across a transmission interface in multipleresolutions and/or multiple streams in a manner that allows forreconstruction, storage and/or display as one or more images on one ormore suitable analog and/or digital video display devices. In thisregard, it will be understood that the actual image content, format,and/or spectral characteristics (e.g., visual image, infrared image,radar image, etc.) of image streams processed and transmitted accordingto the disclosed systems and methods may vary according to the needs ordesired characteristics for a given application, and is independent ofthe implementation of one or more features of the disclosed systems andmethods.

[0048] In one embodiment, the disclosed systems and methods may beemployed for receiving image information having a native resolution andframe rate from one or more image source/s, and for processing andproviding the image information across an analog interface for displayin digital format as one or more images of native or less than nativeresolution and/or of native or less than native frame rate (e.g., as oneor more native high resolution image/s, as one or more images of nativeand/or less than native frame rate, as one images of native or less thannative resolution, as one or more zoomed or non-zoomed images, or as anycombination thereof). In this regard, a single image may be displayed ormultiple images may be simultaneously displayed, e.g., two or moreimages of multiple resolution, frame rate, and/or zoom scale may besimultaneously displayed on a single digital display device (e.g.,computer monitor) or on multiple digital display devices. For example, adesired scene may be transmitted over standard analog interface (e.g.,standard CCTV interface), and displayed simultaneously in multipleresolutions (e.g., windows containing high and standard resolutionsimultaneously).

[0049] In another embodiment, the disclosed systems and methods may beemployed for receiving image information having a native resolution andframe rate from one or more image source/s, and for processing andproviding the image information across an analog interface for displayin analog format as one or more images (e.g., as one or more zoomed orunzoomed images, or a combination thereof). In this regard, a signalimage may be displayed or multiple images may be simultaneouslydisplayed on a single analog display device (e.g. conventional analogcomposite video monitor) or on multiple analog display devices. Forexample, a desired scene may be transmitted over standard analoginterface (e.g., standard CCTV interface), and displayed simultaneouslyin multiple zoom scales (e.g., unzoomed and zoomed windows displayedsimultaneously).

Transmission across Analog Interface for Display on Digital DisplayDevice

[0050] In one embodiment, the disclosed systems and methods may beimplemented with multiple image creation circuitry that may be coupledto create and provide multiple image information (e.g.,multiple-resolution and/or multiple-stream image information) via analogsignals to multiple image processing circuitry for image storage, imagedisplay, and/or further image processing. As an example, FIG. 2illustrates one exemplary embodiment of a video system 200 havingmultiple resolution image capability that may be implemented in thepractice of the disclosed systems and methods, for example, to createand provide multiple-resolution image transmission capability over aconventional analog transmission interface 206, e.g., having a frequencylimit of 4-5 MHz. As shown in FIG. 2, video system 200 may includemultiple stream image creation circuitry in the form of an enhancedresolution digital camera 202 that includes multiple stream imageprocessing components (224, 225, 226, 228, 230, 232), the features ofwhich may be implemented using any suitable hardware and/or softwareconfiguration (e.g., digital signal processor (“DSP”), applicationspecific integrated circuit, field programmable gate array, combinationsthereof, etc.). Also shown in FIG. 2 is multiple stream image receivingcircuitry in the form of a PC-based DVR 204 that includes imagereconstruction and multiple stream processing/storage capabilities. Asshown in FIG. 2, video system 200 may be implemented in one embodimentto provide a multiple resolution image display on a standard PC monitor208 (e.g., simultaneous display of both standard resolution image 209and high resolution image 211).

[0051] Although FIG. 2 illustrates a multiple resolution video system200 capable of providing multiple resolution image capability (e.g.,standard and high resolutions in this exemplary embodiment), it will beunderstood that the disclosed systems and methods may be alternativelyimplemented to provide a video system capable of providing multiple(e.g., two or more, three or more, etc.) resolutions and/or that themultiple image resolutions may be selected to be any combination ofdesired resolutions, e.g. whether the multiple resolutions are selectedto each be high resolutions, each to be standard resolutions, or whethera combination of such resolutions is selected.

[0052] Furthermore, it will be understood that it is not necessary thatthe disclosed systems and methods be practiced with two or more imageresolutions that are not standard and/or high resolution images asdefined herein. In this regard, any two or more multiple resolutions ofdesired different resolutions (i.e., of lower and higher resolutionrelative to each other) may be selected, including selected resolutionsthat are neither standard or high resolution. In this regard, a multiplestream image system having multiple resolution capability may beemployed, for example, in the embodiment described with relation to FIG.2 for creation, transmission and optional display of standard and highresolution images, it being understood that this embodiment is exemplaryonly and that other embodiments of the disclosed systems and methods maybe configured for the creation, transmission and optional display ofthree or more images of differing resolutions, and/or for the creation,transmission and optional display of two or more images with differingresolutions that are not necessarily either standard and/or highresolution images. In addition, it will also be understood that thedisclosed systems and methods need not be practiced to process andtransmit multiple image resolutions, but instead may be implemented toprocess and transmit a single resolution using multiple streams.

[0053] In FIG. 2, enhanced resolution digital camera 202 is showncoupled to PC-based DVR 204 by bandwidth-limited analog interface 206(e.g., similar to the bandwidth-limited analog interface 106 of FIG. 1).As used herein, the term “bandwidth limited interface” is used todescribe a signal transmission format (e.g., standard composite videoanalog transmission interface) that has the characteristic of limitedtransmission capacity relative to the transmission capacity required totransmit a given image signal (e.g., high resolution image signal). Oneexample of a bandwidth limited interface is a typical coaxialcable-based analog video transmission format that is limited toapproximately 485 TV lines of resolution due to the following factors:

[0054] Bandwidth limitations associated with transmission of a levelbased signal (such as analog video) over coaxial cable which will rolloff the higher frequencies as the length of the cable increases.

[0055] Encoding of the luminance and chrominance into a single compositevideo signal which requires frequency attenuation of the luminance datain order for it to not be interpreted as chrominance information.

[0056] Limitations built into existing frame grabber interfaces whichare designed to work with the expected video bandwidths.

[0057] It will be understood that a standard composite video analogtransmission interface is only one example of a bandwidth limitedinterface. Furthermore, although a bandwidth-limited interface isillustrated in FIG. 2, it will be understood that the disclosed systemsand methods may be implemented to provide images over any other type ofinterface/transmission media suitable for analog and/or digital signaltransmission, including interfaces that are not bandwidth-limitedrelative to the desired signal to be transmitted. Other examples ofinterfaces with which the disclosed systems and methods may beadvantageously employed to transmit a desired image signal usingmulti-stream methodology include, but are not limited to, fiber opticinterface formats, wireless (radio frequency) interface formats, serialdigital formats, etc. In this regard, examples of other types ofsuitable interfaces include, but are not limited to, conversion todigital video for transmission over fiber, conversion to the RF domainfor transmission over coaxial cable or direct wireless transmission,etc. It will thus be understood that any type of signal may betransmitted (in bandwidth or in non-bandwidth limited format relative tothe interface), and that additional processing or conversion may beemployed as long as the transmission contains multi-stream informationin accordance with the systems and methods disclosed herein. In the casewhere the interface is non-bandwidth limited the techniques describedherein may be usefully employed, e.g., to reduce the data bandwidth inorder to preserve other system resources such as hard drive storage.

[0058] As illustrated for the exemplary embodiment of FIG. 2, digitalcamera 202 includes optics 220 and an image source in the form of videoor image sensor 222, in this exemplary embodiment a high resolutionvideo sensor having a standard HDTV resolution of 1280×720 (as opposedto the standard 720×480 or 720×576 resolution of the video sensor ofprior art digital camera 102 of FIG. 1). Although a HDTV high resolutionsensor is illustrated and described in relation to the exemplaryembodiment of FIG. 2, it will be understood with benefit of thisdisclosure that a digital camera may be provided with any other typeand/or resolution of high resolution or standard resolution image sensoror combination of such sensors that are suitable for generating imageinformation having characteristics described further herein. Examples ofsuitable image sensors include, but are not limited to, image sensorshaving resolutions of 1920×1080, 1280×1024, 2048×2048, etc. Suitabletypes of image sensors include, but are not limited to, CCD, CMOS,ultraviolet, near infrared, infrared, etc.

[0059] Although video system 200 of the exemplary embodiment of FIG. 2is configured having a camera with an image source in the form of animage sensor, it will be understood that a multiple resolution videosystem may be configured in the form of a camera or in a non-cameraconfiguration with any one or more types of image sources that aresuitable for generating image information in any picture format, e.g.,including image sensors, storage devices capable of providing previouslygenerated and stored image information, high frame rate images generatedfrom RADAR or laser mapping, etc.). Furthermore, an image source may bea digital image source, or may be an analog image source producing ananalog signal that is converted to digital signal via digital to analog(“DAC”) conversion. In one embodiment, the disclosed systems and methodsmay be implemented to simultaneously receive, process and interleaveimage signals from two more image sources, e.g., image signalsoriginating from wide-angle view and narrow angle view image sensors,image signals from IR and visible spectrum image sources, image signalsfrom two image storage devices, etc.

[0060] As shown in FIG. 2, camera 202 may be configured withconditioning circuitry 221 that is capable of converting informationfrom sensor 222 into video information having a desired digital videoform. In this regard, conditioning circuitry 221 may be any circuitrysuitable for converting raw sensor information into desired digitalvideo form (e.g., standard format digital video form) including, but notlimited to circuitry capable of converting CCD or CMOS sensorinformation (e.g., the bayer pattern red-green-blue (“RGB”), oralternately cyan-magenta-yellow (“CMY”)) into planar RGB or CMY. Otherpossible tasks which may be performed by conditioning circuitry 221include, but are not limited to, the control of an image sensor toproduce the proper signal levels (exposure control), providing thecorrect synchronization of sensor control signals, and otherconditioning aspects such as aperture correction (i.e. high frequencyboost), white balance, color correction, demosaicing, dynamic rangeprocessing, dynamic range compression, formatting, etc. Color spaceconversion such as RGB or CMY to 4:4:4, 4:2:2 or 4:2:0 format YCrCb mayalso be performed by circuitry 221. In one exemplary embodiment,conditioning related processing may be performed via embedded digitallogic or via algorithms running on a DSP or any other suitableprocessing element. It will be understood that type/s and combination/sof types of conditioning may vary and may be performed by circuitry 221as desirable to fit the characteristics of a given application. It isalso possible that no conditioning circuitry may be present, forexample, in the case of a digital video signal received from a videosource in the desired signal form (e.g., received in the form of astandard format digital video signal).

[0061] As illustrated for the exemplary embodiment of FIG. 2, multiplestream image creation circuitry in the form of camera 202 may beconfigured with multi-stream image processing circuitry 270 thatincludes Frame store circuitry 224 that stores the higher resolutionframe, one or more instances of Window circuitry 225 that is configuredto extract a selected portion of the higher resolution frame to form afull or windowed partial image, one or more instances of Image Scalercircuitry 226 that scales the higher resolution image down to a NTSC,PAL, SECAM, etc. compatible format or scales a smaller section of thehigher resolution frame up to NTSC/PAL compatible formats for outputover one of the output interfaces 206, Image Deconstruction circuitry228 that segments or “Tiles” the high resolution image into multipletile segments (e.g., four pieces), Alignment Data Insert circuitry 230that inserts alignment data into the unused lines of the tiled image,and Image Mux circuitry 232 that selects the information from separatesources (e.g., either the high resolution image scaled down to NTSC/PALresolutions, a high resolution tile, or zoomed segment of a highresolution image) and transmits it in a multiple stream (e.g., dualstream) format. In this regard, it will be understood that multipleinstances of window circuitry 225 and image scaler circuitry 226 may beprovided in order to implement separate image data paths that may bemultiplexed for transmission in a manner as will be described furtherherein, for example, in relation to FIG. 8A. Although not illustrated,it is also possible to have multiple instances of other circuitry withinmulti-stream image processing circuitry 270 (e.g., circuitry 228,circuitry 230, etc.) as necessary or desired to meet the requirements ofa given application.

[0062] Also shown in FIG. 2 is optional supplemental image processingcircuitry 223 that may be implemented as shown for purposes ofimplementing algorithms such as facial or target recognition andadvanced motion detection algorithms. In such an exemplary embodiment,these types of algorithms may be provided with access to full frame rateand full resolution video in it's highest quality form, e.g., before anychrominance downsampling to 4:2:2 and/or any degradation going to theanalog domain and back again. Such an implementation may be employed asan alternative or in addition to implementation of such algorithms inmultiple stream image receiving circuitry (e.g., such as PC-based DVR204) after transmission of analog signals across interface(s) 206.

[0063] In the illustrated embodiment, optional supplemental imageprocessing circuitry 223 may be any circuitry suitable for receivinghigher resolution frames from a suitable image source, in this caseconditioning circuitry 221, and for implementing one or moresupplemental image processing algorithms/circuitry that is sensitive topixel resolution (i.e., sensitive to the number of pixels in an image)prior to providing frames to Frame store circuitry 224. In this regard,pixel resolution-sensitive algorithms/circuitry refers to anyalgorithms/circuitry that is affected by degradation or loss of pixelsfrom an original image (e.g., due to image processing in multi-streamimage processing circuitry 270 or other processing that results in lossof pixels). Specific examples of pixel resolution-sensitivealgorithms/circuitry include, but are not limited to, facial, licenseplate, or target recognition algorithms and circuitry used to implementsame; motion detection algorithms and circuitry used to implement same;etc. In this regard, increased pixel resolution may facilitaterecognition of movement, features, etc. within an image prior to furtherprocessing. Further, increased frame rate of an original image (e.g.prior to processing that reduces frame rate) also may benefit operationof such algorithms/circuitry. Examples of suitable circuitryconfigurations that may be employed for optional supplemental imageprocessing circuitry 223 include, but are not limited to, digital logiccontained within an FPGA, ASIC, or standalone integrated circuit (“IC”)or algorithms running on a DSP or other suitable processing element,etc.

[0064] In the illustrated embodiment, Frame store circuitry 224 may beany circuitry suitable for storing higher resolution frames receivedfrom a suitable image source, in this case conditioning circuitry 221(or from optional supplemental image processing circuitry 223 whenpresent), and for providing these frames to Window circuitry 225/ImageScaler circuitry 226 and/or Image Deconstruction circuitry 228 as may beappropriate for a given application or operating mode of same. In thisregard, Frame store circuitry 224 may be configured to store or stagehigh resolution frames and then provide them to Image Deconstructioncircuitry 228 where high resolution image frames are segmented (orseparated into tiles) by circuitry 228. Furthermore, Frame storecircuitry 224 may be configured with bypass circuitry so that highresolution frames received from conditioning circuitry 221 or optionalsupplemental image processing circuitry 223 may be directly provided toWindow circuitry 225/Image Scaler circuitry 226 without frame storage.Alternatively a switch or other suitable selection mechanism (not shown)may be provided with corresponding signal paths that together areconfigured to selectably route high resolution frames to either of Framestore circuitry 224 or directly to Window circuitry 225/Image Scalercircuitry 226 without frame storage. Examples of suitable circuitryconfigurations that may be employed for Frame store circuitry 224include, but are not limited to, SDRAM or SRAM either standalone orcontained within an ASIC or FPGA, etc.

[0065] Window circuitry 225 and Image Scaler circuitry 226 may be anycircuitry configurations suitable for respectively extracting portionsof a higher resolution image and scaling frames of a higher resolutionimage obtained from Frame store circuitry 224 (or alternatively receiveddirectly from conditioning circuitry 221 or from optional supplementalimage processing circuitry 223 when present) to frames of a lowerresolution image (e.g., NTSC, PAL, SECAM, etc. compatible format), andfor providing these scaled lower resolution frames to Image Muxcircuitry 232. Alternately selected areas may be scaled up for largerdisplay on the monitor 208 via one of the transmission interfaces 206.The advantage of this alternate path is the ability to send higher framerate scaled images at the same time the lower effective frame ratehigher resolution images are being transmitted. Examples of suitablecircuitry configurations that may be employed for Window circuitry 225and Image Scaler circuitry 226 include, but are not limited to, digitallogic contained within an FPGA, ASIC, or standalone IC or algorithmsrunning on a DSP or other suitable processing element, etc.

[0066] Image Deconstruction circuitry 228 may be any circuitry suitablefor segmenting (or separating into tiles) the high resolution imageframes received from Frame store circuitry 224 into two or more multiplehigh resolution segments, performing appropriate buffering operations toprevent latency issues from arising, and for providing the segmentedhigh resolution frames or tiled high resolution images to Alignment DataInsert circuitry 230. Examples of suitable circuitry configurations thatmay be employed for Image Deconstruction circuitry 228 include, but arenot limited to, digital logic contained within an FPGA, ASIC, orstandalone IC or algorithms running on a DSP or other suitableprocessing element, etc.

[0067] Alignment Data Insert circuitry 230 may be any circuitry suitablefor inserting tile identification information, horizontal alignmentinformation, and/or vertical alignment information into the unused linesof the segmented frames or tiled images received from ImageDeconstruction circuitry 228. Examples of suitable circuitryconfigurations that may be employed for Alignment Data Insert circuitry230 include, but are not limited to, digital logic contained within anFPGA, ASIC, or standalone IC or algorithms running on a DSP or othersuitable processing element, etc.

[0068] Image Mux circuitry 232 may be any circuitry suitable forselecting the scaled lower resolution frames from Image Scaler circuitry226 or the higher resolution tile images from Alignment Data Insertcircuitry 230 for transmission using multiple stream (i.e., two or morestream) format. In this regard, Image Mux circuitry 232 may provide theselected image information to conversion circuitry 244. In oneembodiment, Image Mux circuitry 232 may be configured to include digitalbuffering and switching circuitry and may be characterized as a mediarouter. Examples of suitable circuitry configurations that may beemployed for Image Mux circuitry 232 include, but are not limited to,digital logic contained within an FPGA, ASIC, or standalone IC oralgorithms running on a DSP or other suitable processing element, etc.In an alternative embodiment, analog multiplexing may be used. It willbe understood that Image Mux circuitry 232 may be configured to outputmultiple signal streams, e.g., to multiple buffers and/or DAC circuitsfor transmission across one or more analog interfaces 206.

[0069] As illustrated in FIG. 2, multiple instances of conversioncircuitry 244 may be optionally provided to output separate signalstreams (e.g. each being multiple or single streams themselves) toseparate respective analog interfaces. It will be understood, however,that multiple interfaces 206 need not be present however, and that asingle analog interface 206 may be employed. It will further beunderstood that one or more digital transmission interfaces (e.g., suchas described elsewhere herein) may alternatively be employed and, in oneembodiment, may be employed in combination with one or more analogtransmission interfaces. Use of multiple transmission interfaces may beemployed to allow increased frame rate and/or resolution to betransmitted simultaneously. For example, using two analog interfaces 206allows multiple image creation circuitry 202 to transmit a four-tileHDTV image at an effective frame rate of 7.5 fps over a first analoginterface 206 a, while at the same time transmitting a full standarddefinition image at 30 fps over a second analog interface 206 b. In sucha case, the HDTV image and the standard definition image may bedisplayed simultaneously on the same display device, displayed onseparate respective display devices, and/or stored.

[0070] It will further be understood that the disclosed systems andmethods may be advantageously practiced in combination with one or moredigital transmission interfaces, e.g., to reduce bandwidth requirements.In one embodiment, for example, the disclosed systems and methods may beadvantageously implemented in combination with a digital transmissioninterface where bandwidth capacity is an issue, e.g., due to thesupported rate of a particular connection or a particular series ofconnections, due to cost associated with bandwidth, due to a need tomultiplex several video signals over one digital link (e.g., due totopology constraints, cost issues, right of way, physical access andavailability, etc).

[0071] As further illustrated in FIG. 2, one or more embedded processors298 (e.g., multiprocessor, DSP, or other suitable processor/s) may bepresent in camera 202 for implementing one or more tasks (e.g.,algorithms) described herein and related to one or more individualcircuitry components 221, 223, 270 and 244. As shown in FIG. 2, camera202 may also include a conversion block 244 for digital to analogencoding (e.g., conversion from digital video to standard composite NTSCanalog video) for transmission across interface 206.

[0072] Although FIG. 2 illustrates one exemplary embodiment includingmultiple stream image creation circuitry that is provided within acamera 202 it will be understood that it is possible that multiplestream image creation circuitry may be implemented in any other suitableform or configuration, e.g., as circuitry provided within a camera or ascircuitry provided wholly or in-part separate to a camera, as circuitrywith any other suitable configuration of optics and/or image sensor, ascircuitry that is operably coupled for receiving signals from non-cameravideo sources, combinations thereof, etc. Furthermore, it will beunderstood that in other embodiments multiple stream image creationcircuitry (e.g., configured within a camera, configured as separatecircuitry, etc.) may be implemented with other types of image processingcircuitry.

[0073] It will also be understood that multi-stream image processingcomponents may be implemented using any alternative circuitconfiguration suitable for providing any one or more of the respectivecapabilities thereof as described herein. In this regard, one or more ofthe exemplary illustrated components of multi-stream image processingcircuitry 270 (e.g., 224, 226, 232, 228 and 230) may be alternativelyrearranged and/or combined in any alternative configuration suitable forimplementing the functionality described herein, either in relation toeach other, and/or in relation to other circuitry (e.g., such asoptional image processing circuitry 223, optional conditioning circuitry221, conversion circuitry 244, etc.). Furthermore, it is possible thatadditional circuitry components may be provided in the path betweenmultiple image creation circuitry and a coupled bandwidth-limitedtransmission interface as may be needed or desired for other purposeswithout departing from the scope of the disclosed systems and methods.In this regard, examples of such circuitry include, but are not limitedto, image storage circuitry, one or more types of image conditioningcircuitry, one or more types of supplemental image processing circuitry,etc.

[0074] Still referring to the exemplary embodiment of FIG. 2, videosystem 200 is configured with multiple stream image receiving circuitryin the form of PC-based DVR 204. As illustrated in FIG. 2, PC-based DVR204 includes frame grabber circuitry 250 for analog to digitalconversion of analog signals (e.g., multi-stream analog signals)transmitted from enhanced resolution digital camera 202 across interface206. Frame grabber circuitry 250 is shown coupled to bandwidth limitedinterface 206 via multiport interface component 248, although couplingvia any other suitable type of interface component is possible. DVR 204is also shown provided with multi-stream image processing circuitry 290that is complementary to multi-stream image processing circuitry 270 ofcamera 202. In this regard, components of multi-stream image processingcircuitry 290 of the exemplary embodiment of FIG. 2 include ImageReconstruction circuitry 252 that may be used to reconstruct tiled imagesegments back into the original high resolution image, Multi StreamCompression circuitry 254 that may be used to compress both the standardand high resolution images to decrease storage requirements, and MultiImage/Stream Storage circuitry 256 that may be used to store the raw orcompressed images, e.g., based on system or system operatorrequirements.

[0075] As shown in FIG. 2, for this exemplary embodiment ImageReconstruction circuitry 252 and Multi Stream Compression circuitry 254may be implemented together on processor 260 (e.g., PC-basedmicroprocessor such as an Intel Pentium) of PC-based DVR that may be inturn coupled to multiple image storage 256, which may be any datastorage device or combination of data storage devices suitable forstorage of raw or compressed image information, e.g., such as hard orfloppy disk/s, EEPROM, FLASH, DRAM, SRAM, etc. However, it will beunderstood that the configuration of multi-stream image processingcircuitry illustrated in FIG. 2 is exemplary only, and that componentsthereof may be alternatively implemented on more than one processorand/or using dedicated circuitry (e.g., such as PCI-based standalonecompression and image processing cards), either of which that may be inwhole or in part configured external to other components of a DVR.Furthermore, it will be understood that multiple stream image receivingcircuitry may be implemented using non-PC based DVR circuitconfigurations, or using any other configurations of one or morecircuits (e.g., including non-DVR circuit configurations) suitable forproviding any one or more of the respective capabilities of multipleimage processing circuitry as described herein. For example, it will beunderstood that capabilities of a frame grabber component may becombined with multi-stream image processing circuitry, and that amultiple image storage component is not required, e.g., multiple imageinformation may be processed and/or displayed only in real time,multiple image information may be processed and then provided withoutstoring to other and separate image processing circuitry for additionalimage processing, etc.

[0076] In the illustrated embodiment of FIG. 2, frame grabber circuitry224 may be any circuitry suitable for capturing a frame from an analogvideo signal (e.g., multi-stream analog signal) received acrossinterface 206, converting it to a digital image, and providing it tomulti-stream image processing circuitry 290. Examples of suitablecircuitry configurations that may be employed for frame grabbercircuitry 224 include, but are not limited to, industry standard videodecoder IC's coupled to a synchronization and memory controller coupledto SDRAM for video storage, etc.

[0077] Image Reconstruction circuitry 252 may be any circuitry suitablefor reconstructing tiled image segments back into the original highresolution image originally segmented by Image Deconstruction circuitry228 using alignment data inserted by Alignment Data Insert circuitry 230of enhanced resolution digital camera 202. Examples of suitablecircuitry configurations that may be employed for Image Reconstructioncircuitry 252 include, but are not limited to, PC or DSP basedprocessing with or without dedicated hardware assist, etc.

[0078] Multi-Stream Compression circuitry 254 may be any circuitrysuitable for compressing both standard and high resolution imageinformation received across interface 206 and/or processed by imagereconstruction circuitry 252. In the illustrated embodiment, MultiStream Compression circuitry 254 may provide such compressed imageinformation to Multi Stream Storage circuitry 256, for example, todecrease storage space required for the image information in MultiStream Storage circuitry 256. Multi Stream Storage circuitry 256 mayadditionally or alternatively provide compressed image information toother system or subsystem components, e.g., for purposes of storage orsubsequent transmission to other viewing stations via LAN (Local AreaNetwork) or WAN (Wide Area Network). Examples of suitable circuitryconfigurations that may be employed for Multi Stream Compressioncircuitry 254 include, but are not limited to, FPGA or ASIC based logic,dedicated standalone MPEG, JPEG, JPEG2000 integrated circuits, etc.

[0079] In the embodiment of FIG. 2, Multi Stream Storage circuitry 256may be any data storage device suitable for storing lower and higherresolution images, e.g., standard and high resolution images. Someexamples of suitable data storage devices have been previously describedherein. In one exemplary embodiment, lower and/or higher resolutionimages may be selectably stored in either raw and/or compressed form,e.g., based on command or control signals provided by a system operatoror another system or another subsystem component. For example, it may bedesirable to store images in raw form in those cases where any lossyform of compression would adversely affect post processing such asfacial recognition, or target detection, or cases where the image is tobe significantly digitally enlarged before analysis (e.g. retrieval of alicense number off of a car tag that does not represent a significantportion of the overall image, personnel identification either by zoomingin on a face of other aspects of the persons apparel, etc.).Alternatively, images may be stored in compressed form such as when itis desired that the amount of data be decreased in order to reducestorage requirements (e.g., to reduce cost or to increase the amount ofimage time span that may be stored) or to enable a higher frame rateover slower communications links (e.g., modem, ISDN, etc.) than would bepossible with non-compressed video.

[0080] In the embodiment of FIG. 2, multiple resolution image display208 is illustrated as a single display device (e.g., computer monitor orother suitable digital image display device) configured tosimultaneously display standard resolution image 209 and higherresolution image 211 received from DVR 204. However, it will beunderstood that in the practice of the disclosed systems and methodsthat three or more multiple images of different resolutions and/or framerates may be simultaneously displayed on a single display device in realtime and/or in playback form from recorded data. It will also beunderstood that two or more multiple images of different resolutionsand/or frame rates may be displayed on multiple display devices (e.g.,standard resolution image displayed on a first display device and highresolution image simultaneously displayed on a second display device) inreal time and/or in playback form from recorded data. Furthermore, itwill be understood that display of multiple images is not necessary inthe practice of the disclosed systems and methods. For example, one ormore images may be displayed in real time while other non-displayedimage information is further processed or recorded for future playbackor processing. Alternatively, it is possible that no image is displayedin real time, with all image information being recorded and/or processedby other separate image processing circuitry.

[0081] In one embodiment of the practice of the disclosed systems andmethods, a multiple resolution video system (e.g., such as illustratedand described in relation to FIG. 2) may be implemented in a manner thatallows image information received from one or more image sources (e.g.,image sensor/s, storage device/s containing previously generated andstored image information, etc.) to be selectably processed in a mannerso as to create two or more image streams based thereupon that havedifferent resolutions and/or frame rates. In this regard, frames of eachcreated image stream may correspond to an entire image received from animage source or may correspond to any portion thereof. In this regard,one or more whole or partial images may be segmented into two or moretile segments for transmission across an analog interface. Multipleimages (e.g., higher and lower resolution images) and/or tile segmentsthereof may be simultaneously transmitted across an analog interface bytransmitting each of the multiple images at respective frametransmission rates that together do not exceed the maximum frametransmission rate capacity of the interface.

[0082] For example, in one example implementation of the embodiment ofFIG. 2, image sensor 222 may provide high resolution image informationrelating to a scene viewed by sensor 222 to multi-stream imageprocessing circuitry 270 of camera 200. Multi-stream image processingcircuitry 270 of enhanced resolution digital camera 202 may in turngenerate a standard resolution image frame of the entire scene from thehigh resolution image information (e.g., using circuitry related toimage scaling, segmentation, and multiplexing). This standard resolutionimage frame may then be transmitted via digital to analog conversioncircuitry 244 to DVR 204 across interface(s) 206 that has a maximumframe rate transmission capacity (e.g., 30 frames per second) at a firstframe rate (e.g., 15 frames per second) that is less than the maximumframe rate capacity of interface 206. Simultaneously, higher resolutionsegmented image tiles associated with respective portions of the highresolution image scene may also be transmitted in analog form acrossinterface 206 by multi-stream image processing circuitry components 270in camera 202 via conversion circuitry 244 to DVR 204 at a second framerate (e.g., 15 frames per second) that is also less than the maximumframe rate capacity of interface 206, and such that the first and secondframe rates do not together exceed the maximum frame rate transmissioncapacity (e.g., 30 frames per second) of interface 206.

[0083] Once received by DVR 204, frame grabber 250 may provide both thestandard resolution and high resolution image information in digitalform to multi-stream image processing circuitry 290 of DVR 204.Microprocessor-based image processing circuitry 252 may then beconfigured to reassemble and display the scene of the standardresolution image at its transmitted frame rate (e.g., 30 frames persecond) and/or to display the high resolution scene assembled from thesegmented tiles at a frame rate corresponding to the transmission rateof the individual tiles divided by the number of tile segments perimage. For example, a high resolution image segmented into four tilesegments that are transmitted across interface 206 at a rate of 15frames per seconds may be reassembled into a single high resolutionimage that may be processed and/or displayed at a rate of 3.75frames/sec. In an alternative example, a high resolution-only mode maybe implemented that allows for transmission of four high resolutionframes at a total rate of 30 frames per second from multi-stream imageprocessing circuitry components 270 of camera 202 to multi-stream imageprocessing components 290 of DVR 204, for reassembly into a single highresolution image at a rate of 7.5 frames/sec, i.e., four frames that areassembled to make up each overall frame of the full image aretransmitted every 7.5 seconds.

Tile Reassembly to Form Reconstructed Image

[0084] In a further exemplary embodiment of the disclosed systems andmethods, two or more multiple image tiles may be transmitted separately(e.g., across standard CCTV composite video interface or other suitablestandard or non-standard interface/s) and may be reassembled into areconstructed image (e.g., high resolution full 1280×720 image). Forexample, in one exemplary embodiment, such a methodology may beimplemented in multi-resolution video system 200 of FIG. 2 to transmitmultiple high resolution image tiles (e.g., each high resolution imagetile being mapped into a respective standard TV resolution video frame)between camera 202 and DVR 204 across analog interface 206 forreassembly into a reconstructed full high resolution image with littleor no visible artifacts. However, it will be understood that such amethodology may be implemented in any other single ormultiple-resolution video system environments having any otherconfiguration of one or more circuit blocks suitable for performing oneor more of the video signal processing tasks described herein.

[0085] In those embodiments where transmission of high resolution images(e.g., 1280×720 images) exceed the capacity of the selected analogtransmission media interface (e.g., NTSC, PAL, etc.), tiled highresolution images may be transmitted across the analog interface at aframe rate that is equal to the frame rate of the analog interfacedivided by the number of tiled images transmitted. For example, NTSCtransmission media is limited to 720×480 rectangular pixel (“RP”) videoframes at 29.97 fps, and PAL transmission media is limited to 720×576rectangular pixels (“RP”) video frames at 25 fps. Thus, where four highresolution tiled images are transmitted across the selected analoginterface, the resultant effective frame rate of the reassembled highresolution image may be one-fourth the actual frame transmission rate(e.g., 30/4 or 7.5 fps for NTSC standard, or 25/4 or 6.25 fps for PALstandard).

[0086] It will be understood that the capture/archive rate of multiplestream imaging circuitry may not equal the full frame rate of thecoupled transmission interface. For example, referring to the embodimentof FIG. 2, PC-based DVR 204 may not capture/archive frames at the fullframe rate of analog interface 206 (e.g., 30 fps NTSC, 25 fps PAL).Therefore, without compensation, there may be mismatches between thecapture rate of DVR 204 and the timing of the received frames containingthe tiled high resolution images. For example, a DVR capturing at a 2fps rate would capture/archive one out of every 15 frames for an NTSCtransmission, and one out of every 12.5 frames for a PAL transmission.In an embodiment where a single high resolution image is transmitted bymultiple image stream circuitry in the form of four tiled images in arepeating 1-2-3-4 pattern, there may therefore be a mismatch between themultiple stream image creation circuitry and the coupled multiple streamimage receiving circuitry. In one exemplary embodiment, such mismatchesmay be compensated for by adapting the frame generation pattern of themultiple stream image processing circuitry (e.g., camera 202) to complywith the capture/archive rate of the coupled multiple stream imagereceiving circuitry (e.g., DVR 204).

[0087] The following table illustrates one exemplary manner (i.e., forNTSC standard) in which compensation may be implemented by repeatingtransmission of each frame containing a given image tile repetitivelyback-to-back manner for a sufficient number of times to allowcapture/archive of at least one frame containing the given image tilebased on the capture/archive rate of the multiple image stream receivingcircuitry, it being understood that similar methodology may be employedfor PAL standard and that other compensation scheme methodologies arealso possible. Frame Transmission Repetition Rate (Number DVR NTSC ofTimes A Given Frame is Resulting Effective Capture Rate RepeatedlyTransmitted Reassembled Image Rate (fps) Back to Back) (fps) 30 fps 17.5 15 2 3.75 10 3 2.5

[0088] The disclosed systems and methods may be advantageously employedto transmit and reassemble tiles in a manner that takes advantage ofextra or left-over information space that exists when a image having afirst resolution is transmitted using a format that has a secondresolution that is greater (e.g., greater number of pixels and/orgreater number of rows) than the first resolution. For example,referring to the exemplary video system 200 of FIG. 2, an exemplary1280×720 square pixel HDTV sensor 222 of system 200 may be employed incombination with exemplary and conventional video encoder/decoder partsthat are based on rectangular pixel formats corresponding to CCIR-656video streams. For these encoder/decoder parts the resolution of theNTSC image is about 720×480. It will also be understood that 720×480rectangular TV pixels (“RP”) corresponds to approximately 640×480 squarepixels (“SQP”).

[0089] For the combination of this given example, a 1280×720 imagegenerated by the HDTV sensor 222 may be divided by Image Deconstructioncircuitry 228, for example, into four 640×360 tiles for transmissionusing the NTSC format. Thus, insertion of each 640×360 high resolutiontile into a standard NTSC format image leaves 80 horizontal pixels andan additional 120 lines available for alternate (e.g., non-video)purposes, such as described below. It will be understood that thisspecific combination of 640×360 tiles with NTSC format is exemplaryonly, and that the disclosed systems and methods may be implemented withany combination of tile size that is smaller relative to thetransmission frame size (e.g. PAL format, etc.), such that one or morelines and/or horizontal pixels are available for alternate purposes asfurther described herein.

[0090]FIGS. 4A-4C illustrate one exemplary embodiment as it may beemployed in the reassembly of four 640×360 tiles 402, 404, 406 and 408that have been inserted for transmission in four respective NTSC frames422, 424, 426 and 428, e.g., by Image Deconstruction circuitry 228 ofsystem 200 or other suitable circuitry. As shown in FIG. 4A, each tile402, 404, 406 and 408 may be vertically centered with a vertical linealignment pattern 410 inserted into one of the available lines precedingthe active image, e.g., by Alignment Data Insert circuitry 230 of system200 or other suitable circuitry configuration. In the illustratedembodiment of FIG. 4A, each vertical alignment pattern is also shownprovided with an optional tile identifier pattern 416 therein, e.g.,also inserted by Alignment Data Insert circuitry 230 of system 200 orother suitable circuitry configuration. In such an embodiment, avertical line alignment pattern 410 with tile identifier pattern 416 maybe used to uniquely identify each tile for reassembly with other tilesinto a larger image 400, e.g., as upper left quadrant tile of the largerimage, upper right quadrant tile of the larger image 400, etc. Inaddition, the placement of the alignment pattern 410 in the video streammay be used for proper vertical alignment and reassembly of tile withadjacent tile/s, e.g., by Image Reconstruction circuitry 252 of videosystem 200 or other suitable circuitry configuration. For example, avertical alignment pattern 410 may be defined such that the active imageof each 640×360 tile starts at a fixed number of lines (e.g., 16 lines)from the video line that this vertical alignment pattern occupies. Inanother exemplary embodiment, vertical blanking information area, suchas that associated with closed caption, may alternatively oradditionally be used to indicate that the field/frame is a “tile”associated with a larger image, as well as to indicate the relativespatial relationship of the tile with respect to the original largerimage.

[0091] It will be understood that the foregoing examples of verticalalignment patterns are exemplary only, and that other configurations ofvertical alignment information and/or tile identifier information (e.g.,configured as one or more other types of patterns and/or codes) may beemployed that are suitable for vertical alignment and/or identificationof a given tile with respect to other tiles of a given image. Forexample, a tile identifier pattern or other identifier information maybe employed that is separate from a vertical alignment pattern or othervertical alignment information employed for the same tile.

[0092] Also shown in the exemplary embodiment of FIG. 4A is how eachtile 402, 404, 406 and 408 may be configured to support horizontalalignment information, in this example in the form of horizontalalignment patterns 412 e.g., inserted by Alignment Data Insert circuitry230 of system 200 or other suitable circuitry configuration. In theillustrated embodiment, a horizontal alignment pattern 412 may beconfigured, for example, to occur temporally to the right of each of thetiles associated with the left of the original image and to the left ofeach of the tiles associated with the right of the original image.

[0093] One exemplary horizontal alignment pattern may be as simple as aset of four 3 pixel wide pulses (pure white pixels on a blackbackground) that may be used (e.g., by Image Reconstruction circuitry252 of video system 200 or other suitable circuitry configuration) toallow the alignment of the segments or tiles horizontally, e.g., viasimple correlation. For example, as shown in FIG. 4B, a horizontalalignment correlation value may be obtained by horizontally shiftinglines 412 a and 412 b of respective horizontally adjacent tiles relativeto each other (as indicated by the arrow in FIG. 4B) and performing astandard correlation function 450. In such a case, the peak correlation452 occurs when the pulses 430 a and 430 b for horizontally adjacenttiles are at maximum alignment with respect to each other. Thismethodology may be used to take into account the sampling jitter thatmay occur when the incoming analog image is being re-sampled by a videodecoder chip over multiple frame times.

[0094] In a further exemplary embodiment, each tile 402, 404, 406 and408 may also be configured to include a small horizontal “overlap” area414, e.g., consisting of 16 pixels, e.g., by Alignment Data Insertcircuitry 230 of system 200 or other suitable circuitry configuration.Once the tiles 402, 404, 406 and 408 are properly aligned usinghorizontal alignment patterns 412 via the horizontal alignment techniquedescribed above, the respective overlap areas 414 b and 414 a ofhorizontally adjacent tiles (e.g., tiles 402 and 404, and tiles 406 and408) may be combined (e.g., using fading transfer function 440 andsummation and limiting function 442) as indicated in FIG. 4C to form thefinal larger image 400 shown in FIG. 4D. This combination may beperformed, for example, by Image Reconstruction circuitry 252 of videosystem 200 or other suitable circuitry configuration. It will beunderstood that techniques other than the simple linear approximationapproach illustrated may be employed, e.g., non-linear techniques suchas square root or logarithmic, or techniques that take into account theimage contrast around the combined area such as adaptive filtering.

[0095] It will be understood that the foregoing examples of horizontalalignment patterns and methodology are exemplary only, and that otherconfigurations of horizontal alignment information (e.g., configured asone or more other types of patterns and/or codes) may be employed thatare suitable for horizontal alignment and/or identification of a giventile with respect to other tiles of a given image. It will also beunderstood that alignment and overlap blending may also occur verticallyto allow for the case where the tiles are first compressed and storedand then later retrieved, decompressed, and reassembled. This allows forany visual artifacts that may occur due to the compression technique atthe image boundaries to be masked. If vertical blending is performedthen the tile identifiers may also be used for alignment purposes.

[0096] To maximize resolution and bandwidth availability, it will beunderstood that the tile reassembly techniques described herein may beextended to utilize more of the used pixel area through the properapplication of image scaling. For example, the 640 image may behorizontally scaled to 704 before transmission, which still allows forthe 16 pixels required for the pattern and overlap area shown in FIGS.4B and 4C.

[0097] It will be understood that the forgoing example utilizing1280×780 pixel HDTV resolution is exemplary only, and that other imagesegmentation and reconstruction configurations may be employed to fit agiven application. For example, a 1920×1080 HDTV resolution image may bedivided into six 640×480 tiles and reconstructed into a single image ina manner similar to described above.

[0098] Image Signal Processing for Transmission Across Bandwidth LimitedInterface

[0099]FIG. 8A illustrates one exemplary embodiment of image data logicflow as may be implemented, for example, using multi-stream imageprocessing circuitry components of camera 202 of FIG. 2 or using othersuitable configuration of multi-stream image processing circuitry. Inaddition to the exemplary circuitry embodiment of FIG. 2, it will beunderstood that the image data logic flow illustrated in FIG. 8A may beimplemented with any alternative circuitry configuration suitable foraccomplishing one or more of the image data processing tasks describedin relation thereto.

[0100] As shown in FIG. 8A, a digital image data 833 may be receivedfrom any suitable image source 860 and then travel down one of fourexemplary paths before being selected by the frame multiplexingoperations 863 for final processing for transmission across abandwidth-limited interface in a manner as previously described. Thisexemplary video flow may be employed, for example, to support thevarious selectable display modes of Examples 1-4 described herein. Inthis embodiment, multiple windows and scaling functions may beimplemented to provide for multiple zoomed or non-zoomed images whichmay then be combined into a single image (or frame) for analogtransmission or sent as sub-images within subsequent frames. Thesewindowing and scaling mechanisms may be implemented by logic containedwithin an FPGA or ASIC, or alternatively may be performed by a DSP orother suitable high speed processing element.

[0101] In the illustrated embodiment of FIG. 8A, image source 206 mayprovide digital image data 833 (e.g., full high resolution image data of1280×720 pixels at 30 frames per sec or fps). Image source 860 may beany source of a suitable image, such as is described elsewhere herein.For example, image source 860 may be an image storage device, may be animage sensor that produces raw image data that may be converted intodesired digital video form (e.g., standard format digital video form) byconditioning circuitry, etc. As illustrated, image source 860 mayprovide digital image data 833 for multi-stream image processing 861,e.g., as may be performed by multi-stream image processing circuitry 270of FIG. 2 or other suitable multi-stream image processing circuitryconfiguration. Digital image data 833 may be in any suitable digitalimage data form and, in one exemplary embodiment, may be digital videodata having an image size of 1280×720 square pixels at 30 frames/second(fps), although digital image data 833 may also be of greater than orlesser resolution and/or greater than or lesser frame rate in otherembodiments. Although digital image data 833 may originate from an imagesource such as image sensor 222 of FIG. 2, it will be understood thatdigital image data may be received for multi-stream image processingoperations 861 from any other suitable image source configuration, suchas described elsewhere herein. Also possible is optional additionalimage processing (e.g., such as performed by optional supplemental imageprocessing circuitry 223 of FIG. 2) that may be performed prior toproviding digital image data for multi-stream image processing 861.

[0102] Still referring to the exemplary embodiment of FIG. 8A,multi-stream image processing 861 may be implemented to provide fourpossible processing paths for digital image data 833, which for purposesof this discussion will be assumed to represent a high resolution imageof a desired scene. Selection of one or more of these paths may beaccomplished, for example, using a switch, data router, or othersuitable selection mechanism 829 provided within circuitry configuredfor performing multi-stream image processing operations 861. Forexample, referring to multi-stream image processing circuitry 270 ofFIG. 2, bypass circuitry may be provided within frame store circuitry224 or as separate switchable data provided from conditioning circuitry221/optional supplemental image processing circuitry 223 to either oneof frame store circuitry 224 or window circuitry 225/image scalercircuitry 226.

[0103] In first data path 801, digital image data 833 may be downscaledin downscaling operations 862 (e.g., by image scaling circuitry 226 ofFIG. 2) to produce downscaled video data 835 of the entire desired scenein standard resolution, e.g., an image scene size of 640×360 squarepixels at 30 fps, for transmission as data 835 across analog interface206 of FIG. 2 after frame multiplexing operations 863 that may beperformed, for example, by image mux circuitry 232. In such anembodiment, when received and processed by multiple stream imageprocessing circuitry (e.g., PC-based DVR 204), data 835 may be displayed(e.g., on image display 208 of system 200), for example, as a 640×360standard resolution image of the entire desired scene at 30 fps.

[0104] In second data path 803, frames of the entire desired scene ofdigital image data 833 may be stored or staged in frame store operations864 (e.g., by frame store circuitry 224 of FIG. 2) and then provided indata path 837 for image deconstruction and insertion of alignment dataat 865 (e.g. image deconstruction circuitry 228 and alignment datainsert circuitry 230 of FIG. 2). In this regard, an entire frame of thedesired scene or other type of image may be segmented into multipletiles (e.g., four quadrants), and appropriate alignment data may beinserted at 864 for image reconstruction purposes and the resultingtiled image information provided as tiled image data 839 (e.g., an imagetile size of 640×360 square pixels at 30 fps) for transmission as data839 across analog interface 206 of FIG. 2 after frame multiplexingoperations 863. When received and processed by multiple stream imagereceiving circuitry (e.g., PC-based DVR 204 of FIG. 2), data 839 may bedisplayed (e.g., on image display 208 of FIG. 2), for example, as areconstructed 1280×720 high resolution image of the entire desired sceneat 30 fps.

[0105] In third data path 805, a selected portion of the desired sceneor other image may be extracted from frames of digital image data 833during windowing operations 866 (e.g., performed by window circuitry 225of FIG. 2) to form a windowed partial image of the desired scene orother type of image, e.g., an image window size of 640×480 square pixelsat 30 fps, for transmission as data 841 across analog interface 206after frame multiplexing operations 863. When received and processed bymultiple stream image receiving circuitry (e.g., PC-based DVR 204), data841 may be displayed (e.g., on image display 208 of FIG. 2), forexample, as a 640×480 image of the entire desired scene at 30 fps.

[0106] In fourth data path 807, a selected portion of the desired sceneor other image may be extracted from frames of digital image data 833during windowing operations 867 (e.g., by window circuitry 225 of FIG.2) to form a windowed partial image of the desired scene or other image,e.g., an image window size of 320×240 square pixels at 30 fps, which isrepresented by windowed partial image data 810 in FIG. 8A. The windowedpartial image of data 810 may then be upscaled during upscalingoperations 868 (e.g., by image scaler circuitry 226 of FIG. 2) to form azoomed partial image of the desired scene or other type of image, e.g.,a 2× zoomed image window size of 640×480 square pixels at 30 fps, fortransmission as data 845 across analog interface 206 after framemultiplexing operations 863. It will be understood that in oneembodiment, the amount of magnification of a zoomed image window may becontrolled by the size of the portion of the desired scene that isselected for extraction in windowing operations 867 prior to upscalingoperations 868 (e.g., when upscaling to a fixed image size in upscalingoperations 868). However, it is also possible to control the amount ofupscaling (e.g., by varying the size of the upscaled image producedduring upscaling operations 868). When received and processed bymultiple stream image receiving circuitry (e.g., PC-based DVR 204 ofFIG. 2), data 845 may be displayed (e.g., on image display 208 of FIG.2), for example, as a 1280×960 zoomed image of the entire desired sceneat 30 fps. It is also understood that less multiple smaller segments ofthe larger image may be transmitted to 204 of FIG. 2 via 850 andindividually scaled up to larger images at 204.

[0107] During frame multiplexing operations 863 (e.g., performed byImage Mux circuitry 232), data is selected for transmission (e.g., data835, 839, 841 and/or 845) and may be output as selected image data 850for further appropriate buffering and analog encoding for transmissionacross one or more analog interfaces (e.g., analog interface 206 of FIG.2). In this regard identity of the multiple streams transmitted aredetermined by selection of data 835, 839, 841 and/or 845 during framemultiplexing operations 863.

[0108] It will be understood that the illustrated embodiment of FIG. 8Ais exemplary only, and that any other combination of different numberand types of image streams and stream operations may be implemented. Inthis regard, the number of data paths may be greater or lesser thanfour, and any given data path may represent any one or more window orscaling operations suitable for achieving desired image characteristics.Furthermore, although not illustrated in FIG. 8A, it will be understoodthat the frame rate of any given image data path may vary as may bedesired in order to meet the needs of a given application (e.g., toallow transmission of the desired amount of image information within thebandwidth of an analog interface.

Tile Identifier Pattern Insertion and Detection

[0109] As previously described, image video signal 839 of FIG. 8A maycontain tiled image information (e.g., two or more tiles) fortransmission across an analog interface/s to suitable circuitry forreconstruction or reassembly of the multiple tiles into a single image.Such tiled information may be created, for example, by imagedeconstruction circuitry 228 and alignment data insert circuitry 230 ofFIG. 2 or by any other suitable configuration of one or more circuitsusing, for example, the signal processing flow described in relation tosignal 839 of FIG. 8A or any other suitable signal processingmethodology.

[0110]FIG. 9 illustrates one exemplary embodiment of a representative640×480 de-interlaced active frame 900 having a vertical line alignmentpattern 410 that has been inserted into one of the available linespreceding the active image 910. As previously described, verticalalignment pattern 410 may also be provided with an optional tileidentifier pattern 416. In this regard, FIGS. 11A-11D illustrateexamples of tile identifier patterns 416 such as they may be employed inthe practice of one exemplary embodiment of the disclosed systems andmethods. FIG. 9 also shows right tile overlap area 912 and left tileoverlap area 914 as they may be together present when frame 900 is acenter tile, e.g., of a six tile image. It will be understood that onlyone of right tile overlap area 912 or left tile overlap area 914 need bepresent when frame 900 is either a right tile or left tile, e.g., of afour tile image. As shown in FIG. 9, vertical alignment pattern 410 withoptional tile identifier pattern 416 may also be present below activetile area 910 in order to serve the purpose of vertical blending of theupper and lower tiles. As further shown, non-active tile areas 913 maybe black.

[0111] With regard to horizontal overlap area format definition, activetile area 910 of frame 900 of FIG. 9 has been segmented from an original1280×720 image. In order to split the 1280×720 image into four separate640×360 tiles and reconstruct the original image without presenting anoticeable seam or other visible artifact an overlap area 912 or 914 maybe defined that is 8 pixels wide, i.e., defined on the right border of aleft tile or defined on the left border of a right tile. This overlaparea 912 or 914 may contain 4 pixels that are repeated for both the leftand right tiles, and 4 pixels that are used as a horizontal alignmentmechanism 412. Alternatively, both overlap areas 912 and 914 may bedefined on respective left and right borders in the case of a centertile.

[0112] Still referring to FIG. 9, it will be understood that additionaland/or alternative types of tile identification information may beprovided to further identify individual tile segments of an originalimage and/or to synchronize transmission and receipt of a givenidentified tile across an interface, e.g., to synchronize or coordinatecamera circuitry 202 with frame grabber circuitry 250 of video system200 of FIG. 2, such as under conditions where camera circuitry 202 andframe grabber circuitry 250 are turned on at different times. In oneexemplary embodiment, additional lines of frame 900 may be employed tocontain additional tile identifier information (not shown in FIG. 9) todistinguish sequential transmitted frames containing respectivesequential tiled images from each other, such as four sequentiallytransmitted frames of a four-tile image such as shown in FIG. 4A. Forexample, line 28 may be transmitted alternately as full white and fullblack to indicate frame transmission sequence, e.g., line 28 may be fullwhite to indicate that a given frame 900 is transmitted as frame numberone or as frame number three, and line 28 may be full black to indicatethat a given frame 900 is transmitted as frame number two or as framenumber four. Upon receipt of each frame 900 (e.g., by frame grabber250), line 28 may be examined to determine whether the given frame is afirst or third transmitted frame, or is a second or fourth transmittedframe. It will be understood that additional identification informationmay be provided to further identify or specify the sequential identityof a given frame 900, e.g., line 27 may be alternately full white andfull black in combination with line 28 to specify whether a given frame900 is transmitted as frame zero (e.g., lines 27 and 28 both white), one(e.g., line 27 white, line 28 black), two (e.g., line 27 black, line 28white) or three (lines 27 and 28 both black).

[0113]FIG. 13 illustrates one exemplary image reconstruction embodimentthat may be implemented to reconstruct or reassemble an image frommultiple tiles transmitted across an analog interface according to thedisclosed systems and methods. The methodology of FIG. 13 may beimplemented, for example, using multiple stream image receivingcircuitry in the form of DVR 204, or using any other circuitryconfiguration suitable for image reconstruction/reassembly. Asillustrated in FIG. 13, an incoming analog video signal (e.g., analogvideo signal received across analog interface 206 from camera 202)containing image tiles of a desired video scene may be received andconverted from analog to digital signal form in frame grab step 890(e.g., by frame grabber circuitry 250 or other suitable circuitry). Inoptional selection step 899, image tiles from frame grab step 890 may beselected for further processing, or alternatively, stored image tilesfrom an image storage source (e.g., image storage 256 or other suitableimage storage device) may be selected for further processing.

[0114] Still referring to FIG. 13, tile identifier information of themultiple image tiles associated with a given multi-tile image selectedin step 899 may be detected in step 891, and gathered together in step892 (e.g., by Image Reconstruction circuitry 252 of video system 200).In the event that one or more tiles are received out of sequentialorder, they may be re-queued for processing as shown in step 898.

[0115] Next, the gathered multiple tiles of step 892 may be horizontallyaligned for assembly into the full multi-tile image in step 893 usinghorizontal alignment information included in the individual multipletiles. In optional step 894, the gathered multiple tiles may bevertically aligned for assembly into the full multi-tile image based onoptional vertical alignment information that may be included in thetiles. In step 895, tiles received from step 893 (or from optional step894 when present) may be horizontally blended together, followed byoptional vertical blending of the tiles when appropriate in optionalstep 896. The blended (reconstructed) image may then be displayed (e.g.,on image display 208 or other suitable display device/s) in step 897. Itwill be understood that the particular steps and sequence of stepsillustrated and described in relation to FIG. 13 are exemplary only, andthat other sequences, and/or additional or fewer steps may be employed.For example, horizontal alignment and blending steps 893 and 894 may besequentially interchanged with vertical alignment and blending steps 895and 896, etc.

[0116] With regard to the image reconstruction embodiment illustratedand described in relation to FIG. 13, FIG. 10 illustrates placement of atile alignment pattern 410 with tile identifier pattern 416 when tworespective tiles 1020 and 1022 are vertically assembled to form avertical frame 1024, for example, by image reconstruction circuitry 252of FIG. 2 or by any other suitable configuration of one or morecircuits. As shown, a tile alignment pattern 410 and tile identifierpattern 416 may be present above and below the active tile areas 1010 ofeach of tiles 1020 and 1022, with a vertical blanking area 1030 beingprovided above the active tile areas 1010 of each of tiles 1020 and1022. A horizontal blanking area 1032 may be present to the left of theactive tile areas 1010 of each of tiles 1020 and 1022. These blankingareas may be optionally employed to contain other information such ascamera ID, time stamp, etc.

[0117] Referring again now to FIGS. 9 and 11A-11D, following is anexemplary sequence of steps that may be employed to verify that frame900 of FIG. 9 represents a high resolution tile instead of normalNTSC/PAL video, and to determine which tile segment (e.g., upper left,upper right, lower left, lower right) is contained therein:

[0118] Step 1: Verify lines 0 thru 28 and lines 31 thru 59 of frame 900are full black;

[0119] Step 2: Verify lines 29 and 30 of frame 900 contain one of thepatterns 1002, 1004, 1006 or 1008 of FIG. 11, e.g., via correlation;

[0120] Step 3: While performing step 2, store which pattern (i.e., 1002,1004, 1006 or 1008) was detected on lines 29 and 30; and

[0121] Step 4: Analyze the pattern detected in step 3 to determine thetile placement.

[0122] With regard to Step 4 of the above sequence, exemplary patterns1002, 1004, 1006 or 1008 of respective FIGS. 11A-11D each containinformation that identifies both vertical and horizontal positioning ofthe tile segment of frame 900 within an image assembled from four tilesegments. In this regard, a left pattern portion 1102 and a rightpattern portion 1104 may be provided for each pattern. As illustrated,either the left pattern portion 1102 or the right pattern portion 1104of each tile may be provided as an alternating white and black patternto indicate horizontal positioning of the given tile, i.e., whether thegiven tile is a left or right tile segment. For example, patterns 1002and 1006 each have right pattern portions 1104 that repeatedly alternatebetween full white and full black to indicate that the tile is a rightpattern. In the same manner, patterns 1004 and 1008 each have leftpattern portions 1102 that repeatedly alternate between full white andfull black to indicate that the tile is a left tile segment. Verticalpositioning of a given tile may be indicated by the remaining patternportion that is not used to indicate the horizontal positioning of thetile. For example, patterns 1002 and 1004 each have remaining patternportions, 1102 and 1104 respectively, that have a non-broken full whitepattern to indicate that the given tile is an upper tile. Patterns 1006and 1008 each have remaining pattern portions, 1102 and 1104respectively, that alternate twice between full white and full black toindicate that the given tile is a lower tile.

[0123] It will be understood that the illustrated patterns and patternconfigurations of FIGS. 9-11 are exemplary only and that any otheralignment and/or identifier pattern or other form of alignment and/oridentifier information may be employed that is suitable for aligningadjacent tiles and/or identifying placement of a given tile within agiven reconstructed image. For example, other forms of suitableidentifier information include, but are not limited to identifier code,time stamp, camera identification, camera position in the case of a PTZcamera, etc. Other forms of suitable alignment information include, butare not limited to, square wave patterns, single pulse, ramp, smoothcurve, etc. Furthermore, it will be understood that alignment and/oridentifier information may be adaptive, meaning that the pattern can becommanded to change based on the type of compression method that is usedor how stable the frame grabber interface is.

[0124] Since the standard DVR decoder produces 640×480 images, theresultant reconstructed image horizontally will be less than 1280pixels. For example, in this described four-tile embodiment, a total of8 pixels are used for alignment purposes and a total of 8 pixels areused to create a blended 4 pixel output. Therefore, the resultant imagehorizontal resolution will be 1268 pixels (i.e., 1280 total pixels—8pixels alignment—4 pixels lost due to overlap). In terms of horizontalpixels numbered from 0 to 639 for each left and right tile, thehorizontal structure of the reconstructed image of this embodiment maybe correlated to pixels 0 to 1279 of the original image andcharacterized as follows: Left Tile (upper or lower): Pixel 0-631 =pixels 4 thru 635 of the original image Pixel 632-635 = pixels 636 thru639 of the original 1280 image Pixel 636 = alignment pattern pixel 0 =full black Pixel 637 = alignment pattern pixel 1 = full white Pixel 638= alignment pattern pixel 2 = full black Pixel 639 = alignment patternpixel 3 = full white Right Tile (upper or lower): Pixel 0 = alignmentpattern pixel 0 = full black Pixel 1 = alignment pattern pixel 1 = fullwhite Pixel 2 = alignment pattern pixel 2 = full black Pixel 3 =alignment pattern pixel 3 = full white Pixel 4-7 = pixels 636 thru 639of the original 1280 image Pixel 8-639 = pixels 640 thru 1271 of theoriginal image

[0125] Although implementation of a horizontal overlap area is describedin this example, it will be understood that an overlap area may beprovided at one or more edges of a given video frame adjacent one ormore respective borders of a tile segment contained in said given videoframe, regardless of orientation (e.g., horizontal overlap area,vertical overlap area, etc.).

[0126] It will be understood that the preceding text describes oneexemplary embodiment of the disclosed systems and methods that may beemployed in the separation of an image into multiple (e.g., four)individual or separate tiles, and in the reconstruction of the imagefrom the multiple separate tiles. It will be understood that thisembodiment is exemplary only, and that other tile alignmentconfigurations, tile segment sizes, original image sizes and/or numberof tile segments may be employed to segment an original image into twoor more multiple tile segments and to reconstruct at least a portion ofthe original image by assembling the multiple tile segments together.

Transmission across Bandwidth Limited Analog Interface for Display onAnalog Display Device

[0127]FIG. 3A illustrates another embodiment of the disclosed systemsand methods employing multiple resolution image creation circuitry thatmay be coupled to create and provide multiple image information inmultiple resolutions via analog signals for display on one or moreanalog display devices. In this regard, FIG. 3A illustrates oneexemplary embodiment of a video system 2000 having multiple resolutionimage capability that may be implemented in the practice of thedisclosed systems and methods, for example, to create and providemultiple-resolution image transmission capability over a conventionalanalog transmission interface 2060.

[0128] As shown in FIG. 3A, video system 2000 may include multipleresolution image creation circuitry in the form of a digital camera 2020that includes multiple resolution image processing components (2702,2704, 2706, 2708, 2710, 2712, 2714 and 2716), the features of which maybe implemented using any suitable hardware and/or software configuration(e.g., digital signal processor (“DSP”), application specific integratedcircuit, field programmable gate array, combinations thereof, etc.).Also shown in FIG. 3A is DVR 2040 coupled between analog interface 2060and analog display device 2080, in this case a standard analog displaymonitor 2080. Note that the DVR is not necessary for proper display onthe monitor 2080. As shown in FIG. 3A, video system 200 may beimplemented in one embodiment to provide a multiple resolution imagedisplay on monitor 2080 (e.g., simultaneous display of an entirestandard resolution scene 2082 and three selected zoomed areas 2084,2086 and 2088 of the scene in the area beneath the entire scene 2082).

[0129] Although FIG. 3A illustrates a multiple resolution video system2000 capable of providing a particular multiple resolution imagecapability (e.g., in this exemplary embodiment simultaneous display ofentire non-zoomed scene at standard resolution with three zoomedresolution areas displayed below the entire scene), it will beunderstood that the disclosed systems and methods may be alternativelyimplemented to provide a video system capable of providing anycombination of zoomed and/or non-zoomed resolution images. For example,one or more zoomed areas may be simultaneously displayed with anon-zoomed area, or two or more zoomed areas may be simultaneouslydisplayed alone (i.e., without simultaneous display of non-zoomed area).Furthermore, the particular size of any given zoomed or non-zoomed arearelative to the size of other images that are simultaneously displayedmay be varied as so desired, e.g., three zoomed areas may be displayedin a space occupying greater than half of the overall display spacesimultaneously with a non-zoomed image that occupies less than half ofthe overall display area. In addition, positioning of any given zoomedor non-zoomed area relative to positioning of other images that aresimultaneously displayed in the same display space may also be varied asso desired, e.g., three zoomed areas may be simultaneously displayedabove a non-zoomed area. Thus, it will be understood that the particularcombination of zoomed and/or non-zoomed images simultaneously displayedon a common display, as well as the relative positioning and sizethereof, may be varied (e.g., varied in real time or on a pre-determinedbasis) as needed or desired to meet the requirements of a givenapplication.

[0130] In FIG. 3A, enhanced resolution digital camera 2020 is showncoupled to DVR 2040 and analog display device 2080 by analog interface2060 (e.g., similar to the analog interface 106 of FIG. 1). In thisregard, analog interface 206 may be any signal transmission formatsuitable for transmitting analog video signals. In one exemplaryembodiment, analog interface 206 may be a typical coaxial cable-basedanalog video transmission format (e.g., standard composite videotransmission interface). In this embodiment, analog interface is notnecessarily bandwidth limited.

[0131] It will be understood that a standard composite video analogtransmission interface is only one example of a suitable analoginterface. Other examples of analog interfaces which may beadvantageously employed to transmit a desired image signal usingmulti-resolution methodology include, but are not limited to, fiberoptic, RF, etc. It will also be understood that additional processing orconversion may be employed as long as the transmission containsmulti-resolution information in accordance with the systems and methodsdisclosed herein. Also note that in all cases simple viewing of theresultant image can be performed without the use of the DVR 2040.

[0132] As illustrated for the exemplary embodiment of FIG. 3A, digitalcamera 2020 includes optics 220 and an image source in the form of videoor image sensor 222, in this exemplary embodiment a high resolutionvideo sensor having a standard HDTV resolution of 1280×720 (as opposedto the standard 720×480 or 720×576 resolution of the video sensor ofprior art digital camera 102 of FIG. 1). Although a HDTV high resolutionsensor is illustrated and described in relation to the exemplaryembodiment of FIG. 3A, it will be understood with benefit of thisdisclosure that a digital camera may be provided with any other typeand/or resolution of high resolution or standard resolution image sensoror combination of such sensors that are suitable for generating imageinformation having characteristics described further herein. Examples ofsuitable image sensors include, but are not limited to, image sensorshaving resolutions of 1920×1080, 1280×1024, 2048×2048, etc. Suitabletypes of image sensors include, but are not limited to, CCD, CMOS,ultraviolet, near infrared, infrared, etc.

[0133] Although video system 2000 of the exemplary embodiment of FIG. 2is configured having a camera with an image source in the form of animage sensor, it will be understood that a multiple resolution videosystem may be configured in the form of a camera or in a non-cameraconfiguration with any one or more types of image sources that aresuitable for generating image information in any picture format, e.g.,including image sensors, storage devices capable of providing previouslygenerated and stored image information, high frame rate images generatedfrom RADAR or laser mapping, etc.). Furthermore, an image source may bea digital image source, or may be an analog image source producing ananalog signal that is converted to digital signal via digital to analog(“DAC”) conversion. In one embodiment, the disclosed systems and methodsmay be implemented to simultaneously receive, process and interleaveimage signals from two more image sources, e.g., image signalsoriginating from wide-angle view and narrow angle view image sensors,image signals from IR and visible spectrum image sources, image signalsfrom two image storage devices, etc.

[0134] As shown in FIG. 3A, camera 2020 may be configured withconditioning circuitry 221 that is capable of converting informationfrom sensor 222 into video information having a desired digital videoform. In this regard, conditioning circuitry 221 may be any circuitrysuitable for converting raw sensor information into desired digitalvideo form (e.g., standard format digital video form) including, but notlimited to circuitry capable of converting CCD or CMOS sensorinformation (e.g., the bayer pattern red-green-blue (“RGB”), oralternately cyan-magenta-yellow (“CMY”)) into planar RGB or CMY. Otherpossible tasks which may be performed by conditioning circuitry 221include, but are not limited to, the control of a video sensor toproduce the proper signal levels (exposure control), providing thecorrect synchronization of sensor control signals, and otherconditioning aspects such as aperture correction (i.e. high frequencyboost), white balance, color correction, demosaicing, dynamic rangeprocessing, formatting, etc. Color space conversion such as RGB or CMYto 4:4:4, 4:2:2 or 4:2:0 format YCrCb may also be performed by circuitry221. In one exemplary embodiment, such conditioning processing may beperformed via embedded digital logic or via algorithms running on a DSPor any other suitable processing element. It will be understood thattype/s and combination/s of types of conditioning may vary and may beperformed by circuitry 221 as desirable to fit the characteristics of agiven application. It is also possible that no conditioning circuitrymay be present, for example, in the case of a digital video signalreceived from a video source in the desired signal form (e.g., receivedin the form of a standard format digital video signal).

[0135] Not shown in FIG. 3A are optional image processing circuitry 223and embedded processor/s 298, either or both of which may be present incamera 2020 to perform similar tasks as described in relation to camera202 of FIG. 2.

[0136] As illustrated for the exemplary embodiment of FIG. 3A, multipleresolution image creation circuitry in the form of camera 2020 may beconfigured with multi-resolution image processing circuitry 2700 thatincludes Window circuitry components 2702, 2706 and 2710 coupled torespective image scaler circuitry components 2704, 2708 and 2712. Eachof Window circuitry components 2702, 2706 and 2710 is configured toextract a selected portion of the higher resolution frame to form awindowed partial image, and each of image scaler circuitry components2704, 2708 and 2712 is configured to scale the higher resolution imageto a desired image size for inclusion into a NTSC, PAL, SECAM, etc.compatible format. In this regard, it will be understood that the zoomedresolution of a given image may be controlled by varying the extractedwindow size relative to the degree of image scaling between any givenpair of coupled window and image scaler circuitry components (e.g., toachieve a zoomed, non-zoomed, or fractionally zoomed image).

[0137] It will be understood that multiple instances of window circuitrycomponents (e.g., window circuitry components 2702, 2706 and 2710) andrespectively coupled multiple instances of image scaler circuitrycomponents (e.g., image scaler components 2704, 2708 and 2712) may beprovided in order to implement separate image data paths that may bemerged for transmission in a manner as will be described further herein,for example, in relation to FIG. 8B. Furthermore, as shown in FIG. 3A,within a given data path multiple instances of window circuitrycomponents (e.g., multiple window circuitry components 2710) andrespectively coupled multiple instances of image scaler circuitrycomponents (e.g., multiple image scaler components 2712) may be providedin order to produce multiple windowed and scaled image areas within agiven common data path that may be merged for transmission with imagesof other data paths in a manner as will be described further herein, forexample, in relation to FIG. 8B. In the latter case, sequence controlregister circuitry 2714 (e.g., containing window definition registersand corresponding scaling definition registers) may be provided todirect sequential windowing and scaling operations for each of themultiple zoomed image areas created within a common data path, as willbe described further herein in relation to FIG. 8B.

[0138] It will be understood that the number of instances of windowcircuitry components and respectively coupled image scaler componentsmay be varied as needed or desired to fit the requirements of a givenapplication, e.g., to provide more than three separate data paths, toprovide less than three separate data paths, to implement more than onedata paths that each have the capability to produce multiple zoomedimage areas in the same common data path, etc.

[0139] Window circuitry components 2702, 2706 and 2710 and respectiveImage scaler components 2704, 2708 and 2712 may be implemented using anycircuitry configurations suitable for respectively extracting portionsof an image (e.g., higher resolution image) and for scaling the framesof the extracted image to frames having a desired resolution (e.g., toachieve the desired zoom magnitude) and having a resolution that iscompatible with the video transmission interface (e.g., NTSC, PAL,SECAM, etc. compatible format), and for providing these scaledresolution frames to Frame Buffer circuitry 2716. Examples of suitablecircuitry configurations that may be employed for Window circuitrycomponents 2702, 2706 and 2710 and respective Image scaler components2704, 2708 and 2712 include, but are not limited to, digital logiccontained within an FPGA, ASIC, or standalone IC or algorithms runningon a DSP or other suitable processing element, etc.

[0140] Frame Buffer circuitry 2716 may be any circuitry suitable forreceiving and buffering frames from Image scaler components 2704, 2708and 2712, for selectively merging two or more of these received framesinto a composite video frame and for routing these frames, for example,in a manner described further herein in relation to FIG. 8B. In thisregard, Frame Buffer circuitry 2716 may provide the selected imageinformation to conversion circuitry 244 for digital to analog encoding(e.g., conversion from digital video to standard composite NTSC/PAL,etc. analog video) for transmission across interface 2060. In oneembodiment, Frame Buffer circuitry 2716 may be configured to includedigital buffering and switching circuitry and may be characterized as amedia router. Examples of suitable circuitry configurations that may beemployed for Frame Buffer circuitry 2716 include, but are not limitedto, digital logic contained within an FPGA, ASIC, or standalone IC oralgorithms running on a DSP or other suitable processing element, etc.In an alternative embodiment, analog multiplexing may be used. Althoughnot illustrated, it will be understood that Frame Buffer circuitry 2716may be configured to output multiple signal streams, e.g., to multiplebuffers, media bus interfaces, and/or DAC circuits for transmissionacross one or more analog interfaces 2060.

[0141] Although FIG. 3A illustrates one exemplary embodiment includingmultiple stream image creation circuitry that is provided within acamera 2020 it will be understood that it is possible that multipleresolution image creation circuitry may be implemented in any othersuitable form or configuration, e.g., as circuitry provided within acamera or as circuitry provided wholly or in-part separate to a camera,as circuitry with any other suitable configuration of optics and/orimage sensor, as circuitry that is operably coupled for receivingsignals from non-camera video sources, combinations thereof, etc.Furthermore, it will be understood that in other embodiments multipleresolution image creation circuitry (e.g., configured within a camera,configured as separate circuitry, etc.) may be implemented with othertypes of image processing circuitry.

[0142] It will also be understood that multi-resolution image processingcomponents may be implemented using any alternative circuitconfiguration suitable for providing any one or more of the respectivecapabilities thereof as described herein. In this regard, one or more ofthe exemplary illustrated components of multi-resolution imageprocessing circuitry 2700 (e.g., 2702, 2704, 2706, 2708, 2710, 2712,2714, 2716) may be alternatively rearranged and/or combined in anyalternative configuration suitable for implementing the functionalitydescribed herein, either in relation to each other, and/or in relationto other circuitry (e.g., such as optional image processing circuitry223 (not shown), optional conditioning circuitry 221, conversioncircuitry 244, etc.). Furthermore, it is possible that additionalcircuitry components may be provided in the path between multipleresolution creation circuitry and a coupled analog transmissioninterface as may be needed or desired for other purposes withoutdeparting from the scope of the disclosed systems and methods. In thisregard, examples of such circuitry include, but are not limited to,image storage circuitry, one or more types of image conditioningcircuitry, one or more types of supplemental image processing circuitry,etc.

[0143] Still referring to the exemplary embodiment of FIG. 3A, videosystem 2000 is shown configured with optional DVR 2040 coupled betweenanalog interface 2060 and analog display device 2080. In this regard DVR2040 may be optionally present, for example, to record incoming analogimage data and/or for optionally implementing electronic pan-tilt-zoom(“EPTZ”) capability as described further herein in relation to FIG. 8B.It will be understood, however, that the presence of DVR 2040 isoptional and that in other embodiments an analog display device, such asstandard analog monitor 2088, may be directly coupled to an analoginterface (e.g. coaxial cable-based analog interface 2060) for directlyreceiving analog video images from multiple resolution image creationcircuitry such as camera 2020. Alternatively, it is possible that analogvideo images received across an analog interface from multipleresolution image creation circuitry may be recorded by analog and/ordigital recording device for later display, i.e., without real timedisplay of the image information as it is received. It will also beunderstood that analog video recorder devices, additional analogdisplays and other image processing and/or display devices may beoperably coupled to process or display image information received acrossthe analog interface.

[0144]FIG. 8B illustrates one exemplary embodiment of image data logicflow as may be implemented, for example, using multi-resolution imageprocessing circuitry components of camera 2020 of FIG. 3A. In additionto the exemplary circuitry embodiment of FIG. 3A, it will be understoodthat the image data flow illustrated in FIG. 8B may be implemented withany alternative circuitry configuration suitable for accomplishing oneor more of the image data processing tasks described in relationthereto.

[0145] As shown in FIG. 8B, a digital image data 833 may be receivedfrom any suitable image source 860 and then travel down one of threeexemplary paths 8010, 8050, 8070 before being selected during framebuffer/merge operations 8630 for final processing for transmissionacross an interface in a manner as previously described. This exemplaryvideo flow may be employed, for example, to support the display mode ofFIG. 5 described herein. In this embodiment, multiple windows andscaling functions may be implemented to provide for one or more zoomedimages which may be transmitted across an analog interface for displayon an analog display device. These windowing and scaling mechanisms maybe implemented by logic contained within an FPGA or ASIC, oralternatively may be performed by a DSP or other suitable high speedprocessing element.

[0146] In the illustrated embodiment of FIG. 8B, image source 860 mayprovide digital image data 833 (e.g., full high resolution image data of1280×720 pixels at 30 fps). Image source 860 may be any source of asuitable image, such as is described elsewhere herein. For example,image source 860 may be an image storage device, may be an image sensorthat produces raw image data that may be converted into desired digitalvideo form (e.g., standard format digital video form) by conditioningcircuitry, etc. As illustrated, image source 860 may provide digitalimage data 833 for multi-resolution image processing operations 1461,e.g., as may be performed by multi-resolution image processing circuitry2700 of FIG. 3A or other suitable multi-resolution image processingcircuitry configuration. Digital image data 833 may be in any suitabledigital image data form and, in one exemplary embodiment, may be digitalvideo data having an image size of 1280×720 square pixels at 30frames/second (fps), although digital image data 833 may also be ofgreater than or lesser resolution and/or greater than or lesser framerate in other embodiments. Although digital image data 833 may originatefrom an image source such as image sensor 222 of FIG. 3A, it will beunderstood that digital image data may be received for multi-resolutionimage processing operations 1461 from any other suitable image sourceconfiguration, such as described elsewhere herein. Also possible isoptional additional image processing (e.g., such as performed byoptional supplemental image processing circuitry 223 of FIG. 2A) thatmay be performed prior to providing digital image data formulti-resolution image processing 1461.

[0147] Still referring to the exemplary embodiment of FIG. 8B,multi-resolution image processing operation 1461 may be implemented toprovide three possible processing paths for digital image data 833,which for purposes of this discussion will be assumed to represent ahigh resolution image of a desired scene. Selection of one or more ofthese paths may be accomplished, for example, using a 1 to N switch ormux or other suitable selection mechanism 1429 provided within circuitryconfigured to perform multi-resolution image processing operations 1461.It will be understood that the embodiment of FIG. 8B is exemplary onlyand that it is possible that more than three, or less than three, dataprocessing paths may be provided in other embodiments.

[0148] In first data path 8010, digital image data 833 may be downscaledwithout previous windowing in downscaling operations 8620 (e.g., byimage scaler circuitry 2704 of FIG. 3A) to produce downscaled video data8350 of the entire desired scene in standard resolution (e.g., an imagescene size of 640×360 square pixels at 30 fps) and which may then beprovided for merging with other image data as part of merged image data8500 (e.g., by frame buffer circuitry 2716 of FIG. 3A) and thenformatted and converted to suitable analog form (e.g., by conversioncircuitry 244 of FIG. 3A) for transmission across the analog standardvideo interface 206 of FIG. 3A. In such an embodiment, when theanalog-converted image information from downscaled video data 8350 isreceived by analog display device (e.g., standard analog monitor 2080 ofFIG. 3A), the downscaled video data 8350 may be displayed, for example,as a 640×360 standard resolution image of the entire desired scene at 30fps (e.g., as upper 640×360 image 2082 on 640×480 analog image display2080 of FIG. 3A). In such a case, a space of 640×120 is left under image2082 in which other merged image data may be displayed as will befurther described.

[0149] Still referring to first data path 8010 of FIG. 8B, no windowingoperations are performed so that a downscaled image of the entire scenemay be obtained. Regarding the downscaling operations 8620, downscalingmay be performed to take scale the high-resolution input stream 833 andscale it symmetrically into a 640×360 output stream @ 30 frames/sec. Itwill be understood that the use of symmetrical scaling is optional, butmay be employed to retain the visual aspect ratio of the input imagedata 833 while at the same time matching the analog video interfacestandards-based output buffer requirement. In one exemplary embodiment,a scaling ratio of 0.5 may be applied to both the X and Y axes of theoriginal image, thus rendering a ¼ scaled image that is smaller butidentical in proportion to the input images.

[0150] As previously mentioned, the 640×360 frames of image data 8350may then be forwarded to Frame Buffer/merger operations 8630 where theymay be merged into standard-sized frame buffers of 640×480. In oneexemplary embodiment, when so configured, scaled output frame buffersfrom image data 8450 may be merged in Frame Buffer/merge operations 8630with the 640×360 frame buffers of image data 8350 into a standard640×480 output frame buffer such that a composite analog video streammay be composed from both of these video data logic paths fortransmission across a standard analog video interface. Such an exemplaryembodiment may be used to implement the Zoom-Under-Picture (“ZUP”)display capabilities illustrated by zoomed images 2084, 2086 and 2088under downscaled image 2082 displayed on standard analog monitor 2080 ofFIG. 3A (and further illustrated with regard to Example 5 herein). Itwill be understood that the 640×480 frame buffers described herein arestandard TV resolution frame buffers but in a PC format that uses squarepixels. In this regard, it will be understood that Frame Buffer/mergeoperations 8630 may also merge incoming video stream input data from twoor more of data flow logic paths 8350, 8410 and 8450 into standard720×480 frame buffers suitable for direct output on standard compositevideo (TV) media. Frame Buffer/merge operations 8630 (e.g., frame buffercircuitry 2716 of FIG. 3A) may also be configured to adapt and/or mergeincoming video data streams and to route the resultant video outputstreams to one, or more, video media interfaces and conversioncircuitry.

[0151] In second data path 8050 of FIG. 8B, a selected portion of thedesired scene or other image may be extracted from frames of digitalimage data 833 during windowing operations 8660 (e.g., performed bywindow circuitry 2706 of FIG. 3A) to form a windowed partial image ofthe desired scene or other type of image. The windowed partial image ofdata 8100 may then be optionally scaled during optional scalingoperations 8680 (e.g., by image scaler circuitry 2708 of FIG. 3A) toform a zoomed or non-zoomed partial image of the desired scene or othertype of image for optional merging with other image data as part ofmerged data 8500 (e.g., by frame buffer circuitry 2716 of FIG. 3A) andthen formatted and converted to suitable media form (e.g., by conversioncircuitry 244 of FIG. 3A) for transmission across media interface 206 ofFIG. 3A.

[0152] With regard to second data path 8050, a window may be extractedfrom the incoming image data (e.g., 1280×720 at 30 frames/sec) foroutput as an independent video stream. In one embodiment, data path 8050may be implemented to allow a user to select a ‘window’ of the fullresolution image and view this region-of-interest as a separate videostream. This also allows for optional scaling for magnification (i.e.‘zooming’) of the selected window of the full resolution image and/orfor adaptive scaling to fit the resultant images into a frame bufferresolution acceptable for the frame buffer processing logic (e.g.,640×360 or 640×480) of Frame Buffer/merge operations 8630. In oneexemplary embodiment, two or more multiple image data output streams8500 a to 8500 n may be provided, allowing multiple images or imageareas to be provided by circuitry 1461 for transmission to multipledisplay devices (e.g., for simultaneous transmission of different imagestreams across multiple respective transmission interfaces to enablesimultaneous display on multiple respective analog display devices).

[0153] In one exemplary embodiment, an area of 640×480 pixels may beextracted from the upper left quadrant of the full 1280×720 image andthen sent out as buffers (e.g., as an independent image data stream 8500n) on a designated video media processor via the Frame Buffer processorof Frame Buffer/merge operations 8630. In this exemplary embodiment, noscaling is performed in scaling operations 8680, and instead thewindowed image data 8100 may be allowed to pass through scalingoperations 8680 unscaled and in its extracted windowed form. Since suchimages are passed through at their original resolution in such anembodiment (and not scaled by ¼ as described above in relation to oneexemplary embodiment of data path 8010), to meet display resolutionrequirements, this accomplishes the equivalent of a 4× digital zoom. Inthis regard, the 4× zoom factor occurs since 4 times the pixels arepassed through to the video media from the same area of the originalfull resolution image versus the scaling output performed in data path8010 described above.

[0154] Still referring to second data path 8050 of FIG. 8B, anotherexemplary embodiment may be implemented in which a designated 320×240area of the full resolution image is extracted in windowing operations8660 and then passed to scaling operations 8680. In this case, since the320×240 resolution is ¼ of the necessary pixels required for display ona TV, or equivalent device, scaling operations 8680 may be employed toscale the image symmetrically into 640×480 output frame buffers that arereadily displayable on standard media (this results in the equivalent ofan 8× digital zoom factor). In such a case, the original 320×240 imageis 4× the number of pixel data normally used to represent the same areaon standard display media (since it wasn't equivalently downscaled), andby scaling the 320×240 images to 640×480, another 4× of magnification isprovided by scaling each pixel into 4 pixels worth of data. To furtherillustrate image manipulation flexibility that may be realized in thepractice of this embodiment of the disclosed systems and methods, seconddata path 8050 may be alternatively implemented without scalingoperation 8680 in combination with one of the following two exemplarychoices (it being understood that the following are exemplary only andthat a wide variety of other image manipulation variations may beimplemented)

[0155] 1) No scaling of the 320×240 windowed output data 8100 andplacement of the under-sized image buffers into 640×480 output buffersreadily routable by the Frame Buffer logic of Frame Buffer/mergeoperations 8630; or . . .

[0156] 2) Output the undersized 320×240 buffers of windowed output data8100 to Frame Buffer/merge operations 8630 where Frame Buffer logic maymerge the incoming undersized video frame buffers with other video framebuffers (in one exemplary embodiment these may also be undersized) tocreate a resultant video stream data comprised of images from severalinput video stream data paths. Using this methodology, for example, aPicture-In-Picture scenario may be implemented where a small videobuffer from data path 8050 is overlayed on a section of a full screenvideo buffer from another data path, e.g., data path 8010. In anotherexample, the output video stream buffers from data path 8050 may bemerged with the scaled output of data path 8070 (to be described below)to create several video ‘windows’ in composite video image stream.

[0157] In third data path 8070 of FIG. 8B, multiple selected window(e.g., regions of interest) of the desired scene or other image may beextracted from frames of digital image data 833 during multiplewindowing operations 8670 (e.g., by multiple instances of windowcircuitry 2710 of FIG. 3A) to form multiple windowed partial images ofthe desired scene(s) or other image(s) in image data 8102. Each of theseextracted multiple windowed images of image data 8102 may then beoptionally scaled during multiple scaling operations 8682 (e.g., byrespective multiple instances of image scaler circuitry 2712 of FIG. 3A)to form respective zoomed or non-zoomed partial images of the desiredscene (or other type of image) that is contained in image data 8450.Image data 8450 may then be merged with other image data in framebuffer/merge operations 8630 to form part of merged image data 8500(e.g., by frame buffer circuitry 2716 of FIG. 3A) and then formatted andconverted to suitable media form (e.g., by conversion circuitry 244 ofFIG. 3A) for transmission across analog video or other suitable mediainterface 206 of FIG. 3A.

[0158] With regard to third data path 8070, multiple windowingoperations 8670 and multiple scaling operations 8682 may be performed ina cyclic manner. In this regard, windowing operations 8670 may beperformed using respective multiple instances of window definitionregister operations 8071 (e.g., 8071 a to 8071 c) and respective scalingdefinition register operations 8072 (e.g., 8072 a to 8072 c) that may bepresent, for example, if multiple settings are contained in thesewindowing and scaling registers the sequence control register circuitry2714 of FIG. 3A will cycle through these settings to create individuallyscaled versions of part(s) of the original image. Using thismethodology, windowing operations 8670 (e.g. performed by multipleinstances of window circuitry 2710 of FIG. 3A) may be configured to useconfiguration, or control, data setup that is contained in respectiveWindow Definition register sets to determine where, and how much, pixeldata to extract from the incoming full resolution frame buffers. In oneexemplary embodiment, Window Definition Registers sets may be used byall Windowing logic units to identify the location of a ‘window’ and itssize. The register notation may be in any suitable form (e.g., format),but in one embodiment a register notation format may be employed thathas an ‘X axis offset’ register, a ‘Y axis offset’ register’, a ‘X axissize/length’ register, and a ‘Y axis size/length register; these fourregisters comprising the Window Definition Registers and defining where,and how large, an image window is. Although three window definitionregister operations 8071 are illustrated in FIG. 8B, it will beunderstood that there may be any number of Windows Definition registersused for defining various ‘windows’ (i.e., areas to be extracted).

[0159] Scaling operations 8682 may employ a set of Scaling Definitionregister set operations 8072 that each correspond to a respective windowdefinition register set and that define, per each respective input‘window’ video stream of image data 8102, what scaling ratio to employfor generating the resultant output video frame buffers in image data8450. In one exemplary embodiment, Scaling Definition Registers may becommon to all Scaling logic units and may define the scaling ratio forboth the X and Y axes. Similar to multiple windowing operations 8071,multiple Scaling operations 8682 in data path 8070 may be implementedwith multiple scaling definition registers, in contrast to a singlescaling operation that may employ a single scaling definition registers.

[0160] As illustrated in FIG. 8B, Window Definition register operations(8071 a to 8071 c) and respective Scaling Definition register operations(8072 a to 8072 c) are logically linked together. Therefore, for each‘window’ defined to be extracted from image data 833, there is a WindowDefinition register set 8071 and a directly corresponding ScalingDefinition register set 8072. Together these corresponding register setsmay be used to identify a specific video data stream for a particularvideo ‘window’ to be extracted. It will be understood that three windowdefinition register sets and the corresponding three scaling definitionregister sets are exemplary only, and that less than three or more thanthree windows may be defined and scaled in other embodiments using acorresponding number of respective window and scaling definitionregisters. It will also be understood that cases may exist wherewindowing is performed with no scaling, etc.

[0161] In one exemplary embodiment of data path 8070, all of the definedvideo ‘window’ data streams may be frame multiplexed into a single videooutput data stream. This may be done, for example, by also correlatingthe Window Definition register and Scaling Definition register activitywith their respective processing units via a sequential register controllogic unit in Sequential Register Control operations 8074 (e.g., insequence control register circuitry 2714 of FIG. 3A). In this exemplaryembodiment, a Sequential Register Control operation 8074 may be used todetermine when, and how many, video frames are processed for eachdesignated video ‘window’ stream. In this fashion, it may drive ‘n’number of frames per each video ‘window’ in a continuous cyclic fashion.For example, one may assume that a user has configured three videowindows of 212×120. Since the output video rate of the attached videomedia is 30 frames/second, the Windowing and Scaling operations 8670 and8672 may be implemented to render a total of 30 frames/second. Sincethere are three video ‘windows’ defined by the user for this case, thenthese windows may be processed at an effective rate of 10 frames/secondeach to derive a resultant video data stream 8102 of 30 frames/second.Thus the Sequential Control register operation 8074 may be used to drivewhich Windows Definition register sets and corresponding ScalingDefinition register sets are active at each given interval therebydetermining the multiplexing rate, sequence and overall timing.Optionally, the Sequential Register Control operations 8074 may also beimplemented using an optional signaling or control path to the FrameBuffer processing logic of frame buffer/merge operations 8630 to drive acyclic form of frame buffer merging or placement.

[0162] In one exemplary embodiment for creation of multiple zoomed imageareas in data path 8070, Sequential Control Register logic may beimplemented in sequential control register operation 8074 to evenlysubdivide the available frame rate (e.g., 30 frames/second video framerate) among the active ‘windows’ for purposes of windowing operations8670 and scaling operations 8682. Therefore one active zoomed may beimplemented with 30 frames/second (e.g., when a user selects toimplement multiple window operation 8670 and multiple scaling operation8682 to only create one zoomed image area in data path 8070), two activezoomed image areas may be implemented to each get 15 frames/second,three active zoomed image areas may be implemented to each get 10frames/second, and so on. Also, the output video frame buffers may besent to the Frame Buffer processor for merging, ‘underneath or below’,the 640×360 frame buffers from Path 1 into a composite 640×480 framebuffer output stream.

[0163] It will be understood that many other image processing optionsare available using the set of logical operations that comprise datapath 8070 illustrated in the exemplary embodiment of FIG. 8B. Forexample, it is possible that Sequential Register Control operation 8074may be implemented to drive different video ‘windows’ at longerintervals (e.g., up to several seconds) in a contiguous manner so as toemulate ‘camera tours’. In this regard, camera tours are typicallyperformed by taking video streams from several separate cameras and‘cycling’ through the views on a monitor at a fixed rate of ‘n’ secondseach. In another example, it is possible to create many smaller video‘windows’ in windowing and scaling operations 8670 and 8682 and tooutput these as data stream 8450 to the Frame Buffer processor or othercircuitry employed in frame buffer/merge operation 8630 along withproper buffer placement logic such that a video stream which is acomposite of multiple video ‘windows’ may be generated. For example, sixvideo ‘window’ definitions that are sufficiently small (e.g., in the160×120 pixel range) will fit into a 640×480 frame buffer in a ‘2 rowsof 3 windows each’ configuration. Using a standard ‘round-robin’approach, the Sequential Register Control logic of sequential registercontrol operation 8074 may be implemented to update each ‘window’ at aneffective rate of 5 frames/second, though the actual output rate foreach video media may always be at 30 frames/second. The precedingexamples of this paragraph are exemplary only, with it being understoodthat a variety of other image processing methodologies may beimplemented as needed or desired to fit a given application.

[0164] Still referring to FIG. 8B, frame buffer/merge operation 8630 maybe implemented to receive input image data streams (data streams 8350,8410 and 8450), that each consist of frame buffers. Frame buffer/mergeoperation 8630 may then be implemented to perform buffer adaptationand/or merging, where needed or specified, and then to route theresultant image data stream to on or more video media interfaces. Inthis regard, in one exemplary embodiment buffer adaptation and/ormerging may performed under the following three exemplary conditions:

[0165] 1) The frame size of an incoming image data stream may beincompatible with a particular video media interface to which it isspecified to be routed. For example, data path 8010 may be generating a640×360 frame buffer stream, but the backend video media requires a640×480 video stream. In this case, Frame buffer/merge operation 8630may be implemented to either: a) Generate the frame data on-the-fly(i.e. by sending the 640×360 frame buffer out and then generating thefinal 640×120 frame data for transmission), to satisfy the outgoingmedia requirements, or b) Place the incoming 640×360 buffers into640×480 frame buffers before transfer to the video media interface.

[0166] 2) The incoming video frame buffers of a given data path aresignificantly smaller than required by the target video media interface(e.g., such as case where 320×240 input frame buffers are designated tobe transmitted on a video media interface that requires a 640×480 orgreater output frame buffer size). In such a case, the framebuffer/merge operation 8630 may be implemented to provide severalconfiguration options for encapsulating the 320×240 images in 640×480frame buffers. Example of possible encapsulation options include, butare not limited to, center versus top versus bottom justification,border generation, black space generation, etc.;

[0167] 3) Merging may be performed by capturing incoming frame buffersfrom two or more image data streams and placing them into a unifiedbuffer, using specified placement information (that may be configured),such that the incoming streams are merged into one outgoing data stream.For example, 640×360 input frame buffers of data stream 8350 may beplaced at the top of 640×480 frame buffers, and 212×120 input framebuffers of data stream 8450 may be placed sequentially along the bottom640×120 area of the same 640×480 frame buffers. The buffers may then beadapted, if needed (see above), to the outgoing video media interfaceand routed to that media device. It will be understood that any numberof merging methods may be used to created a variety of composite ‘views’in outgoing video streams.

[0168] As mentioned above, in addition to buffer adaptation and merging,frame buffer/merge operation 8630 may be implemented in one exemplaryembodiment to route the resultant image data streams to one or morevideo media interfaces (e.g., video media devices) based onconfiguration parameters. Such configuration parameters may be fixed,user set, or dynamic. In any case, each output image data stream may beconsidered to represent a logical, or virtual, camera output. Thisrouting capability advantageously may be implemented to enable numerousviewing options. For example, multiple data streams may be routed overone video media interface using a time multiplexed method. With multiplevideo media interfaces, any one data streams (e.g., data stream 8350,8410, 8450) may be configured, either statically or dynamically, to berouted to one or more of the video media interfaces. Furthermore, asdescribed below, each transmission interface may be supported as avirtual camera and additionally support the ability to provide aPan-Tilt-Zoom (“PTZ”) functionality for each image data stream in anindependent manner.

[0169] Although not illustrated in FIG. 8B, frame buffer/merge operation8630 may be configured to route the final output image data streams toone or more video media interfaces. It will be understood that thepredominant standards for TV-related media are analog based, and theprimary standards are NTSC (720×480 @ 30 fps) and PAL (720×576 @ 25fps). Additionally, these standards assume interlaced video input. Giventhese standards, standard square pixel-based images may be converted torectangular pixel based images as used by NTSC and PAL. This may beimplemented by scaling the images from 640×480 (or whatever the case maybe) to NTSC or PAL format, and then clocking the data to the varioustransmission video media interfaces in either an interlaced orprogressive manner.

[0170] It will be understood that the illustrated embodiment of FIG. 8Bis exemplary only, and that any other combination of different numberand types of image streams and image stream operations may beimplemented. In this regard, the number of data paths may be greater orlesser than three, and any given data path may represent any one or morewindow or scaling operations suitable for achieving desired imagecharacteristics. Furthermore, although not illustrated in FIG. 8B, itwill be understood that the frame rate of any given image data path mayvary as may be desired in order to meet the needs of a given application(e.g., to allow transmission of the desired amount of image informationwithin the bandwidth of an analog interface.

[0171] In the foregoing discussions, image processing is described thatmay be implemented to allow multiple image streams (e.g., video streams)to be generated at potentially varying resolutions. Also described arescenarios where a given image stream may be a composite of two or moreindividual image data streams. In one embodiment of the disclosedsystems and methods, it is possible to take any of the sub-highresolution ‘windows’ within a given image stream and to control them ina dynamic (i.e., real-time) manner such that one or more of the‘windows’ behaves as an individual PTZ camera. This PTZ methodology maybe implemented in a digital manner to achieve an Electronic PTZ(“EPTZ”)-like or Digital PTZ (“DPTZ”)-like functionality.

[0172] Referring to the embodiments of FIGS. 3 and 8B, any given image‘window’ that is smaller than the full resolution image may be movedaround within the spatial dimensions of the full resolution image. Forexample, a 320×240 video ‘window’ may be moved around significantlywithin a full resolution 1280×720 image. By employing the multipleWindowing and Scaling processing operations of the disclosed systems andmethods as described herein in relation to FIGS. 3 and 8B, ‘windows’ maybe dynamically moved around by updating the register values of theWindowing Definition Register operations 8071 in a real-time fashion.The register values of the Scaling Definition Register operations 8072may also be updated in real-time, thus providing digital zoom, ormagnification, of the images in each selected ‘window’. This methodologymay be implemented in any manner suitable for updating the appropriaterespective window and/or scaling register values to achieve the desiredeffect. One exemplary embodiment is described below, although othermethodologies are possible.

[0173] Conventional mechanical PTZ cameras are typically controlled viacamera control protocols (e.g., Pelco-P, Pelco-D, AutoDome Control Codeprotocol, SpeedDome RS-422/RS-485 protocol, etc.) that are transferredvia serial communications links such as RS-485, RS-422, or RS-232. Thesecamera control protocols are well documented and they enable themanagement of multiple cameras per serial link using serial messagesthat contain unique camera IDs. Camera IDs are also called ‘addresses’,‘device IDs, or ‘camera numbers’, and they uniquely identify thespecific camera on a serial link as the designated target of a cameracontrol message. Each of these protocols enables a set of basic commandssuch as ‘Pan-Right’, ‘Pan-Left’, ‘Tilt-Up’, ‘Tilt-Down’, ‘Zoom-In’(greater magnification), ‘Zoom Out’ (lesser magnification), and so on.

[0174] In the practice of the disclosed systems and methods, a seriallink may be attached to multiple resolution image creation circuitry(e.g., camera 2020 of FIG. 3A) for supporting camera control commandssuch as described in the preceding paragraph. Additionally, multipleresolution image creation circuitry may be configured to emulatemultiple cameras on a serial link by mapping camera IDs/addresses tospecific ‘windows’ in its output image data streams (e.g., data streams8350, 8410 and 8450 of FIG. 8B), which may optionally be further mappedto two or more respective separate multiple image data output streams8500 (e.g., to 8500 a, 8500 b, and 8500 c, respectively) for display ontwo or more separate analog display monitors. FIG. 3B illustrates theimplementation of one such exemplary embodiment having a serial link2310 (e.g., RS-485, RS-422, RS-232, etc.) that couples a PTZ controllerwith joystick 2320 to an optional processor 2330 provided in multipleresolution image creation circuitry 2700 of camera 2020 via serial port2340. As illustrated, processor 2330 is coupled to each window circuitrycomponent (2702, 2706, 2710), and to each image scaler circuitrycomponent (2704, 2708, 2712) in a manner so as to enable selectivecontrol of these components in order to provide the PTZ capabilitiesdescribed further herein.

[0175] When optional multiple image data output streams 8500 areprovided it is possible, for example, to display multiple zoomed areason one analog display device, and then selectably display and control afull screen zoomed image (e.g. of one of the multiple zoomed areasdisplayed on the other monitor) on another analog display device usingcamera control commands mapped to specific windows. However, any othercombination of two or more images on two or more display devices is alsopossible when multiple image data output streams 8500 are provided.

[0176] In one exemplary embodiment, multiple resolution image creationcircuitry such as camera 2020 of FIG. 3A or 3B may be configured tosupport camera IDs/addresses, e.g., such as camera addresses 3 and 4(for illustration purposes). For example, camera 2020 may map cameraID/address 3 to a 320×240 viewing window in an image data streamprovided by second data path 8050 of FIG. 8B and it may map cameraID/address 4 to one of multiple 212×120 zoomed image areas such asdefined by Windowing Definition Register set operation 8071 and ScalingDefinition Register set operations 8072 in third data path 8070 of FIG.8B. Pan Right/Left/Up/Down commands may be then be provided for eithercamera ID 3 or 4, and the corresponding Window Definition Register setsare modified in Windowing Definition Register operations 8071 to emulatecamera panning within the full resolution image of data stream 833. Forexample, FIG. 14 illustrates how x and coordinates 1500 within a windowdefinition register for a particular windowing operation may be variedin one exemplary embodiment to pan and/or tilt a 320×240 window area1520 within a 1280×720 full resolution image 1530, it being understoodthat these particular window and overall image resolution values areexemplary only.

[0177] Using the above methodology, if a Pan command causes a ‘window’to hit the viewing boundary of the full resolution image, the ‘window’may be configured in one embodiment to stop moving (i.e., to ignore Pancommands that are operating in the errant direction), or move into‘black space’. When Zoom commands are received for a ‘virtual’ camera,the corresponding Scaling Definition Register set may be modified toeither increase the scaling factors/ratios (i.e., to Zoom-Out) ordecrease the scaling factors/ratios (i.e., to Zoom-In). In this manner,multiple virtual cameras may be emulated in one embodiment by mappingcamera IDs/addresses in standard PTZ camera control protocols to EPTZfunctionality provided by the Windowing and/or Scaling operations (e.g.,8670 and/or 8682 of FIG. 8B) on a per ‘window’ basis, and such that anychosen ‘window’ is spatially smaller than the source full resolutionimage (e.g., full high resolution image of image data path 833 of FIG.8B).

[0178] It will be understood that the forgoing EPTZ/DPTZ methodology maybe implemented with regard to any given one or more image data path/s ofmultiple resolution image creation circuitry, regardless of the numberand/or type of data paths implemented therein.

EXAMPLES

[0179] The following exemplary and hypothetical examples areillustrative and should not be construed as limiting the scope of theinvention or claims thereof.

Example 1 Selectable Extraction of Higher Rate High Resolution ImageArea Out of a Lower Rate High Resolution Image for Digital DisplayDevice

[0180]FIGS. 5A and 5B illustrate one exemplary embodiment of thedisclosed systems and methods in which a reconstructed high resolutionimage (e.g., 1280×720) of an entire scene 506 of FIG. 5A may bedisplayed at a reduced frame rate (e.g., at 3.75 frames/second)simultaneously with a selected or selectable portion 508 of the scene(e.g., a 640×480 image area) that may be displayed as a high resolutionimage 510 of FIG. 5B at a higher frame rate (e.g., at 15 frames/sec),e.g., displayed together on image display 208 of multi-resolution imagevideo system 200 or on other suitable digital display device/s. In oneembodiment, the high resolution image 510 of the selected portion 508 ofthe scene 506 may be extracted at the camera end prior to transmission,e.g., by image deconstruction circuitry 228 and image scaling circuitry226 of video system 200, or other suitable circuitry.

[0181] In one embodiment, a RS485 interface or other suitable controlinterface may be provided to allow an operator to select the window areafor display of the smaller image at the higher frame rate, e.g., atimage display 208 or at other suitable interface location. In the caseshown in FIGS. 5A and 5B, the selected area is 640×480 and representsthe full sensor high resolution image 510 for the selected specific area508 at any given time. In a further exemplary embodiment, an operatormay be provided with a control interface that allows the operator toslide portion 508 as a selectable window area around the area of fullimage 506 in order to vary the identity of the selected area or portion508 of the scene 506 for display as image 510, e.g., to achieve a 15frames/sec electronic pan-tilt-zoom (EPTZ) function display ofselectable image 510. This windowing function may be provided, forexample, in image deconstruction circuitry 228 of system 200. In oneembodiment, embedded processor 298 (e.g., multiprocessor, DSP or othersuitable processing circuitry) may intercept the serial (i.e., RS-485,RS-422, RS-232, etc.) commands that may originate from a controlinterface (e.g. joystick, keyboard, touch screen) and then direct theimage deconstruction circuitry 228 to selectively sample the overallsensor area. The image scaler circuitry 226 may also be used to eitherexpand or contract (i.e., scale) the selected area to produce thedesired size output image. It will be understood that multiple instancesof both 226 and 228 may be employed to allow for multiple areas atdifferent scaling factors. In such an embodiment, a control interfacemay be configured to select which pair of image scaler circuitry 226 andimage deconstruction circuitry 228 is actively being controlled. Thismay be accomplished, for example, via interpretation of the serialcommands by processing circuitry 298 of system 200.

[0182] Using the above-described technique, one or more selected orselectable higher frame rate areas may be displayed in any suitablemanner, for example, displayed simultaneously with the entire lowerframe rate scene on a separate monitor or other display device, orsimultaneously displayed with the lower frame area on the same monitoror display device (e.g., in a window separate from the entire lowerframe rate scene, or superimposed over a corresponding portion of thelower frame rate scene in the same window).

[0183] In another exemplary embodiment, when a standard resolution CCTVmonitor is employed as part of a system (e.g., as a spot monitor), ascaled image transmitted at a first rate (e.g., at 15 frames/sec) may betemporarily up-scaled to a higher frame rate (e.g., 30 frames/sec) viaframe replication. Furthermore, for display of a HDTV image (i.e.,having 16:9 aspect ratio) on the standard monitor (e.g., having 4:3aspect ratio), a choice may be made to display the image with blankedvertical lines (cinema mode) or by cropping the edges of the images tofit the 4:3 format.

[0184] The advantage of this exemplary embodiment is that it may be usedto allow an operator to maintain an overall situational awareness whilesimultaneously being able to watch high motion rate events.

Example 2 Selectable Extraction of Zoomed High Resolution Image Area Outof a Larger High Resolution Image for Digital Display Device

[0185]FIGS. 5C and 5D illustrate another exemplary embodiment of thedisclosed systems and methods in which a high resolution image (e.g.,1280×720) of an entire scene 502 of FIG. 5C may be displayed (e.g., at3.75 frames/sec) simultaneously with a selected or selectable zoomedportion 504 of the scene that may be displayed as a higher frame rateimage 550 of FIG. 5D (e.g., at 15 frames/sec or other desirable framerate), e.g., displayed together on image display 208 of multi-resolutionimage video system 200 or on other suitable digital display device/s.For example, in the exemplary embodiment of FIG. 5B the windowed area504 represents a selected or selectable 320×240 area (at the sensornative resolution) that is expanded to 640×480 before being transmittedfrom the camera, e.g., by image scaling circuitry of video system 200.As in the embodiment of Example 1, window 504 may be optionally movablein a selectable manner by an operator in real time, e.g., to scan thescene 502 during an alarm event.

[0186] Using the above-described technique, one or more selectablezoomed areas may be displayed in any suitable manner, for example,displayed simultaneously with the entire unzoomed scene on a separatemonitor or other display device, or simultaneously displayed with theentire scene on the same monitor or display device (e.g., in a windowseparate from the entire scene). Selectable zoomed image/s may also bedisplayed in a superimposed manner over a corresponding portion of theunzoomed scene in the same window and, if desired, appropriate imagescaling may be employed so that the entire scene is viewable in eitherof the zoomed or unzoomed portions of the total displayed image.

[0187] The advantage of this exemplary embodiment is that it may be usedto allow an operator to gain more usable detail out of a specific area(e.g., during an alarm event) while simultaneously maintaining visualawareness of a much larger area.

Example 3 Selectable Extraction of Zoomed High Resolution Image Area Outof a Stored Larger High Resolution Image for Digital Display Device

[0188]FIGS. 6A-6C illustrate the significant advantages that may beachieved with stored video content obtained using the higher resolutionimaging of the disclosed systems and methods as compared to existingstandard resolution imaging. In this regard, FIG. 6A represents anstored unzoomed image taken by a video surveillance camera of an overallscene 600 and, for purposes of this example, may be considered to be astandard resolution image, or alternatively a high resolution (e.g.,HDTV) image according to the disclosed systems and methods. Scene 600includes a portion 602 of the scene 600 that corresponds to an event ofinterest (e.g., person of interest, item of interest, activity ofinterest, etc.). FIG. 6B illustrates a 4× digital zoom of portion 602 ofscene 600 that results when starting from a stored unzoomed standardresolution image of scene 600. In contrast, FIG. 6C illustrates theincreased scene detail that is obtained from a 4× digital zoom ofportion 602 of scene 600 that results when starting from an unzoomedHDTV resolution image of scene 600, such as may be provided as describedelsewhere herein across a standard analog interface using the disclosedsystems and methods. In this regard, higher resolution informationcaptured using the disclosed systems and methods may be used to providesubstantially increased detail of a given portion of a video image andto provide greater information as to what was occurring at the timearound an event of interest, even when the video camera is not zoomedinto that given portion of the video image.

Example 4 Simultaneous Monitoring of Zoomed and Unzoomed Images onDigital Display Device

[0189]FIG. 7 illustrates a graphic display 700 (e.g., computer monitor),and shows how the overall area around an aircraft may be monitored in afirst window display 702 (e.g., image display 208 of multi-resolutionimage video system 200 or other suitable digital display device/s) tomaintain overall situation awareness, while one or more zoomed images ofspecific portions of the overall video image may be simultaneouslymonitored to observe one or more specific area/s of activity orinterest. Specifically, FIG. 7 shows how multiple zoomed images (secondand third window displays 704 and 706 of the graphic display 700) may besimultaneously monitored with overall area display 702. It will beunderstood that one or more specific portions of an overall video imagemay also or alternatively be viewed simultaneously and/or sequentiallyon one or more graphic displays (e.g., multiple computer monitors orstandard television monitors) as may be desirable or needed in a givenapplication. The resultant display may be formed from post-transmissionprocessing (e.g. in a PC) or pre-transmission processing (e.g. in thecamera) depending on the which system and method is used.

Example 5 Simultaneous Monitoring of Zoomed and Unzoomed Images onAnalog Display Device

[0190]FIG. 12 illustrates simultaneous display of a primary image 1202of an entire or overall scene and three zoomed image areas 1210, 1212and 1214 (taken from respective locations 1204, 1206 and 1208 of primaryimage 1202 of the entire scene) on an analog display device 1200, suchas conventional analog display device 2080 of FIG. 3A or 3B or othersuitable analog display device. In the illustrated embodiment, displayof the entire scene in primary image 1202 may be used to maintainoverall situation awareness, while zoomed image areas 1210, 1212 and1214 may be simultaneously monitored to observe more specific area/s ofactivity or interest.

[0191] As illustrated, zoomed image areas 1210, 1212 and 1214 do notoverlay the primary image 1202 of the entire scene, but are insteaddisplayed outside the perimeter of the primary image (i.e., withoutoccluding the viewing space of the primary image). In one exemplaryembodiment, primary image 1202 may be displayed so that it occupies theupper 75% of a standard TV resolution video frame (e.g., displayed asupper 640×360 out of total 640×480) displayed on analog display device1200. In this embodiment, primary image 1202 represents the entire imagesource field of view (“FoV”), e.g., of image sensor 222 or othersuitable image source. Because primary image 1202 has a 16:9 aspectratio, it does not completely fill the full vertical image space of astandard TV-resolution video frame (i.e., having a 4:3 aspect ratio)that is displayed on analog display device 1200. As such, one or moreother images may be displayed in the remaining 640×120 space thatremains at the bottom of the TV-resolution video frame. In theillustrated embodiment of FIG. 12, three zoomed image areas 1210, 1212and 1214 are each 212×120 SQP images that are displayed side by sidebeneath primary image 1202 in a zoom-zone under picture (“ZUP”)configuration.

[0192] In the illustrated embodiment of FIG. 12, each of zoomed imageareas 1210, 1212 and 1214 may be, for example, fixed size zoomed areasof selected sections of an original high resolution image prior to beingscaled to 212×120 (e.g., fixed size zoomed areas of an original 1280×720sensor image from image sensor 222 of video system 2000 or othersuitable image source). In one embodiment, zoomed image areas 1210, 1212and 1214 may be selectively zoomed areas (e.g., selected by systemoperator or suitable image control mechanism) of primary image 1202,having locations 1204, 1206 and 1208, respectively, within the primaryimage 1202 as shown. It will be understood that zoomed area/s mayalternatively be of fixed location within primary image 1202 ifselectability is not desired or needed. In any event, whether fixed orselected, zoomed image areas 1210, 1212 and 1214 may be extracted,zoomed, and placed (e.g., by multiple resolution image creationcircuitry 2700 of FIG. 3A or 3B) within a standard TV-resolution videoframe, along with primary image 1202, at the lower section of the videoframe as shown for display on standard analog display device (e.g.,display device 2080 of FIG. 3A or 3B). Further, in one exemplaryembodiment, the number of displayed zoomed image areas, dimensions ofdisplayed zoomed image areas, and/or position of zoomed image areaswithin a given display relative to a primary image may be selectable,either beforehand and/or in real time (e.g., allowing selection of asingle zoomed image area to be centered below primary image 1202, ratherthan display of all three zoomed image areas 1201, 1212, and 1214).

[0193] The amount of zoom magnification magnitude of zoomed image areas1210, 1212 and 1214 may be fixed, and/or in one embodiment may beselectively varied (e.g., in real time). For example, in one exemplaryembodiment, each of zoomed image areas 1210, 1212 and 1214 may have azoom factor that may be selectively varied from about 2× to about 20×magnification, a zoom magnification of about 4× being a default value,although magnification greater than about 20× is also possible.Magnification less than 1× (i.e., fractional magnification) is alsopossible, e.g., entire 1280×720 may be downscaled to a 212×720 imagearea for display as one of zoomed image areas 1210, 1212 or 1214.

[0194] In one exemplary embodiment, all active zoomed image areas (e.g.,zoomed image areas 1210, 1212 and 1214 of FIG. 12) may share a videoscaling unit within hardware scaling circuitry. In this regard, hardwarescaling circuitry may be configured to operate at the appropriate framerate for the transmission standard employed, e.g., 30 fps for NTSC, 25fps for PAL, etc. Thus, the frame rate for each given zoomed image areamay be equal to the total frame rate (e.g., NTSC, PAL, etc.) divided bythe number of active zoomed image areas, as shown in the followingtable: Number Effective NTSC Frame Effective of Active Zoomed Rate perZoomed Image PAL Frame Rate per Image Areas (ZUPs) Area Zoomed ImageArea 1 30 fps   25 fps 2 15 fps 12.5 fps 3 10 fps 8.33 fps

[0195] As previously described, one or more of zoomed image areas 1210,1212 and 1214 may be manipulated as a virtual pan-tilt-zoom (“PTZ”)camera using standard camera control protocol commands, and/or protocolextensions.

[0196] In the preceding example square pixel units have been used todescribe zoomed image areas having 212×120 resolution. It will beunderstood that this resolution may be changed to rectangular pixelformat. Furthermore, it will be understood that any given number of oneor more zoomed image areas may be simultaneously displayed with aprimary image, including more than three zoomed image areas.Furthermore, it will be understood that zoomed image areas may bealternatively displayed in a space above a primary image, and/orsimultaneously in spaces above and below a primary image (e.g., withappropriate size adjustment). Furthermore one or more secondary outputsmay be utilized to provide a higher resolution presentation to one ofthe selected 212×120 images.

[0197] Examples 1-5 describe and illustrate just a few of the manymultiple resolution image combinations that may be realized using thedisclosed systems and methods, with it being understood that othercombinations of image resolutions, image frame rates, numbers ofdisplayed image areas, display formats (e.g., zoomed or unzoomed),display devices, etc. may be implemented as desirable or needed for agiven application using the disclosed systems and methods.

[0198] Various embodiments of the disclosed systems and methods havebeen illustrated and described herein with respect to CCTV technology.However, it will be understood that the disclosed systems and methodsmay be implemented with any video system technology and associatedcomponents in which one or more benefits of the disclosed systems andmethods may be realized or suitably implemented. Furthermore, thedisclosed systems and methods may be advantageously implemented in anyvideo system environment in which one or more benefits of the disclosedsystems and methods may be realized or suitably implemented. Examples ofsuch environments include, but are not limited to, video surveillanceenvironments, television studio environments, televised sports eventenvironments, military targeting or imaging environments, stored imageanalysis environments, etc. Furthermore, it will be understood thatalthough described herein in relation to square pixels, images of thedisclosed systems and methods may be alternatively scaled, displayedand/or processed using rectangular pixels.

[0199] While the invention may be adaptable to various modifications andalternative forms, specific embodiments have been shown by way ofexample and described herein. However, it should be understood that theinvention is not intended to be limited to the particular formsdisclosed. Rather, the invention is to cover all modifications,equivalents, and alternatives falling within the spirit and scope of theinvention as defined by the appended claims. Moreover, the differentaspects of the disclosed systems and methods may be utilized in variouscombinations and/or independently. Thus the invention is not limited toonly those combinations shown herein, but rather may include othercombinations.

What is claimed is:
 1. A method of providing multiple images fortransmission across an analog interface, comprising: receiving at leastone digital image data input stream, said digital image data inputstream containing digital image information; creating at least twodigital image data streams from said at least one digital data inputstream, each of said at least two digital image data streams comprisingat least a portion of said digital image information; merging said atleast two digital image data streams into a common digital image dataoutput stream; converting said common digital image data output streaminto an analog image output stream; and providing said analog outputimage stream for transmission across said analog interface.
 2. Themethod of claim 1, wherein one of said at least two digital image datastreams comprises a first image having a first resolution; whereinanother of said at least two digital image data streams comprises asecond image having a second resolution; and wherein at least one of:said first and second resolutions are different, or said first imagecomprises a different portion of said digital image data input streamthan said second image, or a combination thereof.
 3. The method of claim2, wherein said at least one digital image data input stream comprises adigital video signal received from a digital video source; and whereinsaid method further comprises providing said analog output image streamas an analog video signal for transmission across said analog interface.4. The method of claim 3, further comprising receiving said analog videosignal from across said analog interface, said analog video signalcomprising said first and second images.
 5. The method of claim 4,further comprising displaying said first and second images of saidanalog video signal on an analog display device.
 6. The method of claim4, further comprising: creating a third digital image data stream fromsaid at least one digital data input stream, said third digital imagedata stream comprising at least a portion of said digital imageinformation and having a third resolution; converting said third digitalimage data stream into a second analog image output stream; andproviding said second analog output image stream as a second analogvideo signal for transmission across a second analog interface;receiving said second analog video signal from across said second analoginterface, said analog video signal comprising said third image; anddisplaying said third image of said second analog video signal on asecond analog display device; wherein at least one of: said thirdresolution is different from said first and second resolutions, or saidthird image comprises a different portion of said digital image data, ora combination thereof.
 7. The method of claim 6, wherein said creatingfurther comprises extracting a window area from said original imageinformation and scaling said extracted window area to create said thirdimage as a zoomed image; and further comprising controlling a value ofsaid scaling in real time based at least in part on a command signal, orcontrolling the position of said extracted window area relative to saidoriginal image in real time based at least in part on a command signal,or a combination thereof.
 8. The method of claim 4, wherein said firstand second resolutions are different and wherein said method furthercomprises displaying said first image at said first resolution whilesimultaneously displaying said second image at said second resolution.9. The method of claim 4, wherein said creating further comprises usingscaling to create said first image as a zoomed image prior to said stepof merging; wherein said second image is not a zoomed image; and whereinsaid step of displaying comprises displaying said zoomed first image onan analog display device while simultaneously displaying said secondunzoomed image on said same analog display device.
 10. The method ofclaim 4, further comprising controlling a value of said scaling in realtime based at least in part on a command signal.
 11. The method of claim4, wherein said digital image information comprises an original image;wherein said step of creating said first image comprises extracting awindow area from said original image information to create said firstimage prior to said step of merging; and wherein said step of displayingcomprises displaying said first image on an analog display device whilesimultaneously displaying said second image on said same analog displaydevice.
 12. The method of claim 4, further comprising controlling theposition of said extracted window area relative to said original imagein real time based at least in part on a command signal.
 13. The methodof claim 4, wherein said digital image information comprises an originalimage; wherein said step of creating said first image comprisesextracting a window area from said original image information and thenupscaling said window area to create said first image as a zoomed imageprior to said step of merging; and wherein said step of creating furthercomprises downscaling said original image information to create saidsecond image; and wherein said step of displaying comprises displayingsaid zoomed first image on an analog display device while simultaneouslydisplaying said downscaled second image on said same analog displaydevice, said second image being downscaled such that it does not occupythe full analog display space of said analog display device, and saidfirst image being displayed on at least a portion of said analog displaydevice that is not occupied by said second image.
 14. The method ofclaim 13, wherein said original image information comprises a highresolution digital image.
 15. The method of claim 13, further comprisingat least one of controlling the position of said extracted window arearelative to said original image in real time based at least in part on acommand signal; controlling a value of said upscaling of said extractedwindow area in real time based at least in part on a command signal; ora combination thereof.
 16. The method of claim 4, wherein said at leasttwo digital image data streams comprises at least four digital imagedata streams; wherein a first one of said at least four digital imagedata streams comprises a first image having a first resolution; whereina second one of said at least two digital image data streams comprises asecond image having a second resolution; wherein a third one of said atleast four digital image data streams comprises a third image having athird resolution; and wherein a fourth one of said at least four digitalimage data streams comprises a third image having a third resolution;wherein each of said at least four images either has a resolution thatis different from each of said other at least four images, or comprisesa different portion of said digital image data input stream than each ofsaid other of said at least four images, or a combination thereof;wherein said analog video signal comprises said first, second, third andfourth images; wherein said digital image information comprises anoriginal high resolution image, and wherein said step of creating saidfirst, second and third images comprises extracting respective first,second and third window areas from said original image and thenupscaling said first, second and third window areas to create saidfirst, second and third images as zoomed images prior to said step ofmerging; wherein said step of creating further comprises downscalingsaid original image information to create said fourth image; and andwherein said step of displaying comprises displaying said zoomed first,second, and third zoomed images on an analog display device whilesimultaneously displaying said downscaled fourth image on said sameanalog display device, said fourth image being downscaled such that itdoes not occupy the full analog display space of said analog displaydevice, and said first, second and third images being displayed on atleast a portion of said analog display device that is not occupied bysaid fourth image.
 17. The method of claim 16, further comprising atleast one of controlling the position of at least one of said first,second, or third extracted window areas relative to said original imagein real time based at least in part on a command signal; controlling avalue of said upscaling of at lest one of said first, second or thirdextracted window areas in real time based at least in part on a commandsignal, or a combination thereof.
 18. A method of providing multipleimages for transmission across an analog interface, comprising:receiving at least one digital image data input stream, said digitalimage data input stream containing digital image information; creatingat least two digital image data streams from said at least one digitaldata input stream, each of said at least two digital image data streamscomprising at least a portion of said digital image information, andwherein a first one of said at least two digital image data streamscomprises a first image having a first resolution, and wherein a secondone of said at least two digital image data streams comprises a secondimage having a second resolution; converting a first one of said atleast two digital image data streams into a first analog image outputstream, and converting a second one of said at least two digital imagedata streams into a second analog image output stream; communicatingsaid first analog output image stream across a first analog interface,and communicating said second analog output image stream across a secondanalog interface; receiving said first analog output image stream fromacross said first analog interface, said first analog output imagestream comprising said first image; receiving said second analog outputimage stream from across said second analog interface, said secondanalog output image stream comprising said second image; displaying saidfirst image of said first analog output image stream on a first analogdisplay device; and displaying said second image of said second analogoutput image stream on a second analog display device.
 19. The method ofclaim 18, wherein said first and second resolutions are different, orwherein said first image comprises a different portion of said digitalimage data input stream than said second image, or a combinationthereof.
 20. The method of claim 18, wherein said at least one digitalimage data input stream comprises a digital video signal received from adigital video source; and wherein said method further comprisesproviding said first and second analog output image streams as first andsecond analog video signals for transmission across said respectivefirst and second analog interfaces.
 21. A method of processing digitalimage data, comprising: providing said digital image data, said digitalimage data comprising a full image; processing said digital image datain a first processing operation to create first processed image datacomprising a first image; processing said digital image data in a secondprocessing operation to create second processed image data comprising asecond image; merging said first and second processed image data into acommon merged data stream; converting said merged data stream into acomposite analog video stream containing said first and second images;and providing said composite analog video stream for communicationacross an analog interface; wherein at least one of: said firstprocessed image data has an image resolution that is different from animage resolution of said second processed image data, or said firstprocessed image data comprises a different portion of said digital imagedata than said second processed image data or a combination thereof. 22.The method of claim 21, further comprising controlling at least one ofsaid first or second processing operations in real time based at leastin part on a command signal.
 23. The method of claim 21, furthercomprising: receiving said composite analog video stream containing saidfirst and second images from across said interface; and displaying saidfirst and second images contained in said composite analog video streamon a common analog display device.
 24. The method of claim 23, whereinsaid first processing operation comprises a downscaling operation; andwherein said second processing operation comprises an image windowingoperation, an image scaling operation, or a combination thereof.
 25. Themethod of claim 24, further comprising controlling at least one of saidfirst or second processing operations in real time based at least inpart on a command signal.
 26. The method of claim 23, wherein said firstprocessed image data comprises a downscaled full image and said secondprocessed image data comprises a windowed image, a scaled image, or acombination thereof.
 27. The method of claim 26, further comprising:processing said digital image data in one or more additional processingoperations to create one or more additional processed image datacomprising a third image; merging said one or more additional processedimage data into said common merged data stream; converting said mergeddata stream into said composite analog video stream containing saidfirst, second and one or more additional images; providing saidcomposite analog video stream for communication across said analoginterface; receiving said composite analog video stream containing saidfirst, second and one or more additional images from across saidinterface; and displaying said first, second and one or more additionalimages contained in said composite analog video stream on a commonanalog display device; wherein said second processing operationcomprises an image windowing operation and an image upscaling operation;wherein said second processed image data comprises an extracted andupscaled windowed image; wherein said one or more additional processingoperations comprises an image windowing operation and an image upscalingoperation; wherein said one or more additional processed image datacomprises an extracted and upscaled windowed image; and wherein at leastone of: said one or more additional processed image data has an imageresolution that is different from an image resolution of said first andsecond processed image data, or said one or more additional processedimage data comprises a different portion of said digital image data thansaid first and second processed image data, or a combination thereof.28. The method of claim 26, wherein said step of displaying comprisesdisplaying said downscaled first image on an analog display device whilesimultaneously displaying said windowed or scaled second image on saidsame analog display device, said first image being downscaled such thatit does not occupy the full analog display space of said analog displaydevice, and said second image being displayed in at least a portion ofsaid full analog display space not occupied by said first image.
 29. Themethod of claim 27, wherein said step of displaying comprises displayingsaid downscaled first image on an analog display device whilesimultaneously displaying said extracted and upscaled windowed secondimage and said extracted and upscaled windowed one or more additionalimages on said same analog display device, said first image beingdownscaled such that it does not occupy the full analog display space ofsaid analog display device, and said second image and said one or moreadditional images being displayed in at least a portion of said fullanalog display space not occupied by said first image.
 30. The method ofclaim 26, further comprising: processing said digital image data in oneor more additional processing operations to create one or moreadditional processed image data comprising a third image; providing saidone or more additional processed image data in a second data stream;converting said second data stream into a second composite analog videostream containing said one or more additional images; providing saidsecond composite analog video stream for communication across a secondanalog interface; receiving said second composite analog video streamcontaining said one or more additional images from across saidinterface; and displaying said one or more additional images containedin said second composite analog video stream on a second analog displaydevice.
 31. The method of claim 30, wherein said one or more additionalprocessing operations comprises an image windowing operation and animage upscaling operation; wherein said one or more additional processedimage data comprises an extracted and upscaled windowed image.
 32. Themethod of claim 31, further comprising controlling said one or moreadditional processing operations in real time based at least in part ona command signal.
 33. Multiple resolution image creation circuitryconfigured to receive at least one digital image data input streamcontaining digital information, said multiple resolution image creationcircuitry comprising: multi-resolution image processing circuitryconfigured to: create at least two digital image data streams from saidat least one digital data input stream, each of said at least twodigital image data streams comprising at least a portion of said digitalimage information, merge said at least two digital image data streamsinto a common digital image data output stream; and conversion circuitrycoupled to said multi-resolution image processing circuitry andconfigured to: receive said common digital image data output stream fromsaid multi-resolution image processing circuitry, convert said commondigital image data output stream into an analog image output stream, andprovide said analog output image stream for transmission across ananalog interface.
 34. The multiple resolution image creation circuitryof claim 33, wherein one of said at least two digital image data streamscomprises a first image having a first resolution; wherein another ofsaid at least two digital image data streams comprises a second imagehaving a second resolution; and wherein at least one of: said first andsecond resolutions are different, or said first image comprises adifferent portion of said digital image data input stream than saidsecond image, or a combination thereof.
 35. The multiple resolutionimage creation circuitry claim 33, wherein said at least one digitalimage data input stream comprises a digital video signal received from adigital video source; and wherein said conversion circuitry is furtherconfigured to provide said analog output image stream as an analog videosignal for transmission across said analog interface.
 36. A multipleresolution image creation and display system comprising said multipleresolution image creation circuitry of claim 35; and further comprisingan analog display device coupled to said analog interface and configuredto receive said analog video signal from across said analog interfaceand to simultaneously display said first and second images of saidanalog video signal.
 37. The multiple resolution image creation anddisplay system of claim 36, wherein: said multi-resolution imageprocessing circuitry is further configured to: create a third digitalimage data stream from said at least one digital data input stream, saidthird digital image data stream comprising at least a portion of saiddigital image information and having a third resolution; and saidconversion circuitry is further configured to: receive said thirddigital image data stream from said multi-resolution image processingcircuitry, convert said third digital image data stream into a secondanalog image output stream, and provide said second analog output imagestream as a second analog video signal for transmission across a secondanalog interface; and wherein said multiple resolution image creationand display system further comprises a second analog display devicecoupled to said second analog interface and configured to: receive saidsecond analog video signal from across said second analog interface,said second analog video signal comprising said third image, and displaysaid third image of said second analog video signal; and wherein atleast one of: said third resolution is different from said first andsecond resolutions, or said third image comprises a different portion ofsaid digital image data, or a combination thereof.
 38. The multipleresolution image creation and display system of claim 37, wherein saidmulti-resolution image processing circuitry is further configured to:extract a window area from said original image information and to scalesaid extracted window area to create said third image as a zoomed image;and control a value of said scaling in real time based at least in parton a received command signal, or control the position of said extractedwindow area relative to said original image in real time based at leastin part on a received command signal, or a combination thereof.
 39. Themultiple resolution image creation and display system of claim 36,wherein said first and second resolutions are different and wherein saidanalog display device is further configured to display said first imageat said first resolution while simultaneously displaying said secondimage at said second resolution.
 40. The multiple resolution imagecreation and display system of claim 36, wherein said multi-resolutionimage processing circuitry is further configured to use scaling tocreate said first image as a zoomed image prior to said merging; whereinsaid second image is not a zoomed image; and wherein said analog displaydevice is further configured to display said zoomed first image whilesimultaneously displaying said second unzoomed image.
 41. The multipleresolution image creation and display system of claim 36, wherein saidmulti-resolution image processing circuitry is further configured tocontrol a value of said scaling in real time based at least in part on areceived command signal.
 42. The multiple resolution image creation anddisplay system of claim 36, wherein said digital image informationcomprises an original image; wherein said multi-resolution imageprocessing circuitry is further configured to create said first image byextracting a window area from said original image information to createsaid first image prior to said merging; and wherein said analog displaydevice is further configured to display said first image whilesimultaneously displaying said second image.
 43. The multiple resolutionimage creation and display system of claim 36, wherein saidmulti-resolution image processing circuitry is further configured tocontrol a position of said extracted window area relative to saidoriginal image in real time based at least in part on a received commandsignal.
 44. The multiple resolution image creation and display system ofclaim 36, wherein said digital image information comprises an originalimage; wherein said multi-resolution image processing circuitry isfurther configured to create said first image by: extracting a windowarea from said original image information and then upscaling said windowarea to create said first image as a zoomed image prior to said merging,and downscaling said original image information to create said secondimage; and wherein said analog display device is further configured todisplay said zoomed first image while simultaneously displaying saiddownscaled second image, said second image being downscaled such that itdoes not occupy the full analog display space of said analog displaydevice, said first image being displayed on at least a portion of saidanalog display device that is not occupied by said second image.
 45. Themultiple resolution image creation and display system of claim 44,wherein said original image information comprises a high resolutiondigital image.
 46. The multiple resolution image creation and displaysystem of claim 44, wherein said multi-resolution image processingcircuitry is further configured to: control a position of said extractedwindow area relative to said original image in real time based at leastin part on a received command signal; control a value of said upscalingof said extracted window area in real time based at least in part on acommand signal; or a combination thereof.
 47. The multiple resolutionimage creation and display system of claim 36, wherein said at least twodigital image data streams comprise at least four digital image datastreams; wherein a first one of said at least four digital image datastreams comprises a first image having a first resolution; wherein asecond one of said at least two digital image data streams comprises asecond image having a second resolution; wherein a third one of said atleast four digital image data streams comprises a third image having athird resolution; and wherein a fourth one of said at least four digitalimage data streams comprises a third image having a third resolution;wherein each of said at least four images either has a resolution thatis different from each of said other at least four images, or comprisesa different portion of said digital image data input stream than each ofsaid other of said at least four images, or a combination thereof;wherein said analog video signal comprises said first, second, third andfourth images; wherein said digital image information comprises anoriginal high resolution image; and wherein said multi-resolution imageprocessing circuitry is further configured to: create said first, secondand third images by extracting respective first, second and third windowareas from said original image and then upscaling said first, second andthird window areas to create said first, second and third images aszoomed images prior to said merging, downscale said original imageinformation to create said fourth image; and wherein said analog displaydevice is further configured to display said zoomed first, second, andthird zoomed images while simultaneously displaying said downscaledfourth image, said fourth image being downscaled such that it does notoccupy the full analog display space of said analog display device, andsaid first, second and third images being displayed on at least aportion of said analog display device that is not occupied by saidfourth image.
 48. The multiple resolution image creation and displaysystem of claim 47, wherein said multi-resolution image processingcircuitry is further configured to: control the position of at least oneof said first, second, or third extracted window areas relative to saidoriginal image in real time based at least in part on a command signal;or control a value of said upscaling of at lest one of said first,second or third extracted window areas in real time based at least inpart on a command signal, or a combination thereof.
 49. A multipleresolution image creation and display system, comprising:multi-resolution image processing circuitry configured to: receive atleast one digital image data input stream, said digital image data inputstream containing digital image information, create at least two digitalimage data streams from said at least one digital data input stream,each of said at least two digital image data streams comprising at leasta portion of said digital image information, and wherein a first one ofsaid at least two digital image data streams comprises a first imagehaving a first resolution, and wherein a second one of said at least twodigital image data streams comprises a second image having a secondresolution; conversion circuitry coupled to said multi-resolution imageprocessing circuitry and configured to: convert a first one of said atleast two digital image data streams into a first analog image outputstream, and convert a second one of said at least two digital image datastreams into a second analog image output stream, communicate said firstanalog output image stream across a first analog interface, andcommunicate said second analog output image stream across a secondanalog interface; first display device coupled to said first analoginterface and configured to receive said first analog output imagestream comprising said first image from across said first analoginterface, and to display said first image; second display devicecoupled to said second analog interface and configured to receive saidsecond analog output image stream comprising said second image fromacross said second analog interface, and to display said second image.50. The multiple resolution image creation and display system of claim49, wherein said first and second resolutions are different, or whereinsaid first image comprises a different portion of said digital imagedata input stream than said second image, or a combination thereof. 51.The multiple resolution image creation and display system of claim 49,wherein said at least one digital image data input stream comprises adigital video signal received from a digital video source; and whereinsaid conversion circuitry is further configured to provide said firstand second analog output image streams as first and second analog videosignals for transmission across said respective first and second analoginterfaces.
 52. Multiple resolution image creation circuitry forprocessing digital image data comprising a full image, said multipleresolution image creation circuitry comprising: multi-resolution imageprocessing circuitry configured to: process said digital image data in afirst processing operation to create first processed image datacomprising a first image, process said digital image data in a secondprocessing operation to create second processed image data comprising asecond image, merge said first and second processed image data into acommon merged data stream; and conversion circuitry configured to:convert said merged data stream into a composite analog video streamcontaining said first and second images, and provide said compositeanalog video stream for communication across an analog interface;wherein at least one of: said first processed image data has an imageresolution that is different from an image resolution of said secondprocessed image data, or said first processed image data comprises adifferent portion of said digital image data than said second processedimage data, or a combination thereof.
 53. The multiple resolution imagecreation circuitry of claim 52, wherein said multi-resolution imageprocessing circuitry is further configured to control at least one ofsaid first or second processing operations in real time based at leastin part on a command signal.
 54. A multiple resolution image creationand display system comprising said multiple resolution image creationcircuitry of claim 52, and further comprising: an analog display devicecoupled to said analog interface and configured to: receive saidcomposite analog video stream containing said first and second imagesfrom across said interface; and simultaneously display said first andsecond images contained in said composite analog video stream.
 55. Themultiple resolution image creation and display system of claim 54,wherein said first processing operation comprises a downscalingoperation; and wherein said second processing operation comprises animage windowing operation, an image scaling operation, or a combinationthereof.
 56. The multiple resolution image creation and display systemof claim 55, wherein said multi-resolution image processing circuitry isfurther configured to control at least one of said first or secondprocessing operations in real time based at least in part on a receivedcommand signal.
 57. The multiple resolution image creation and displaysystem of claim 54, wherein said first processed image data comprises adownscaled full image and said second processed image data comprises awindowed image, a scaled image, or a combination thereof.
 58. Themultiple resolution image creation and display system of claim 57,wherein: said multi-resolution image processing circuitry is furtherconfigured to: process said digital image data in one or more additionalprocessing operations to create one or more additional processed imagedata comprising a third image, merge said one or more additionalprocessed image data into said common merged data stream; wherein saidconversion circuitry is further configure to: convert said merged datastream into said composite analog video stream containing said first,second and one or more additional images, provide said composite analogvideo stream for communication across said analog interface; whereinsaid analog display device is further configured to: receive saidcomposite analog video stream containing said first, second and one ormore additional images from across said interface, and simultaneouslydisplay said first, second and one or more additional images containedin said composite analog video stream; wherein said second processingoperation comprises an image windowing operation and an image upscalingoperation; wherein said second processed image data comprises anextracted and upscaled windowed image; wherein said one or moreadditional processing operations comprise an image windowing operationand an image upscaling operation; wherein said one or more additionalprocessed image data comprises an extracted and upscaled windowed image;and wherein at least one of: said one or more additional processed imagedata has an image resolution that is different from an image resolutionof said first and second processed image data, or said one or moreadditional processed image data comprises a different portion of saiddigital image data than said first and second processed image data, or acombination thereof.
 59. The multiple resolution image creation anddisplay system of claim 57, wherein said analog display device isfurther configured to display said downscaled first image whilesimultaneously displaying said windowed or scaled second image, saidfirst image being downscaled such that it does not occupy the fullanalog display space of said analog display device, and said secondimage being displayed in at least a portion of said full analog displayspace not occupied by said first image.
 60. The multiple resolutionimage creation and display system of claim 58, wherein said analogdisplay device is further configured to display said downscaled firstimage while simultaneously displaying said extracted and upscaledwindowed second image and said extracted and upscaled windowed one ormore additional images, said first image being downscaled such that itdoes not occupy the full analog display space of said analog displaydevice, and said second image and said one or more additional imagesbeing displayed in at least a portion of said full analog display spacenot occupied by said first image.
 61. The multiple resolution imagecreation and display system of claim 57, wherein: said multi-resolutionimage processing circuitry is further configured to: process saiddigital image data in one or more additional processing operations tocreate one or more additional processed image data comprising a thirdimage, provide said one or more additional processed image data in asecond data stream; wherein said conversion circuitry is furtherconfigured to: convert said second data stream into a second compositeanalog video stream containing said one or more additional images,provide said second composite analog video stream for communicationacross a second analog interface; and wherein said multiple resolutionimage creation and display system further comprises a second analogdisplay device coupled to said second analog interface and configuredto: receive said second composite analog video stream containing saidone or more additional images from across said second analog interface,and display said one or more additional images contained in said secondcomposite analog video stream.
 62. The multiple resolution imagecreation and display system of claim 61, wherein said one or moreadditional processing operations comprise an image windowing operationand an image upscaling operation; wherein said one or more additionalprocessed image data comprises an extracted and upscaled windowed image.63. The multiple resolution image creation and display system of claim62, wherein said multi-resolution image processing circuitry is furtherconfigured to control said one or more additional processing operationsin real time based at least in part on a received command signal.
 64. Animage processing and display system comprising multiple resolution imagecreation circuitry coupled to at least one analog display device by atleast one analog interface.
 65. The image processing and display systemof claim 64, wherein said multiple resolution image creation circuitrycomprises multi-resolution image processing circuitry that comprises atleast one window circuitry component, at least one image scalercircuitry component, and at least one frame buffer circuitry component.66. The image processing and display system of claim 64, furthercomprising at least two analog devices, each of said at least two analogdisplay devices being coupled to said multiple resolution image creationcircuitry by a separate respective analog interface.
 67. The imageprocessing and display system of claim 65, further comprising a commandcircuitry coupled to said multi-resolution image processing circuitryconfigured to provide one or more command signals to control at leastone of said at least one window circuitry component, at least one imagescaler circuitry component, or a combination thereof.
 68. The imageprocessing and display system of claim 67, wherein said command signalscomprise camera control commands.
 69. A method of providing multipleimages for transmission across a digital interface, comprising:receiving at least one digital image data input stream, said digitalimage data input stream containing digital image information; creatingat least two digital image data streams from said at least one digitaldata input stream, each of said at least two digital image data streamscomprising at least a portion of said digital image information; mergingsaid at least two digital image data streams into a common digital imagedata output stream; providing said common digital image data outputstream for transmission across said digital interface.
 70. The method ofclaim 69, further comprising receiving said common digital image dataoutput stream from across said digital interface, said common digitalimage data output stream comprising said first and second images; andstoring said first and second images, displaying said first and secondimages on a display device, or a combination thereof.
 71. Multipleresolution image creation circuitry configured to receive at least onedigital image data input stream containing digital information, saidmultiple resolution image creation circuitry comprising multi-resolutionimage processing circuitry configured to: create at least two digitalimage data streams from said at least one digital data input stream,each of said at least two digital image data streams comprising at leasta portion of said digital image information, merge said at least twodigital image data streams into a common digital image data outputstream; and provide said common digital image data output stream fortransmission across a digital interface.
 72. A multiple resolution imageand display system comprising said multiple resolution image creationcircuitry of claim 71; and further comprising a digital processingdevice coupled to said digital interface and configured to receive saidcommon digital image data output stream from across said digitalinterface and to store said common digital image data output stream, todisplay one or more images contained in said common digital image outputstream, or a combination thereof.